diff --git a/.github/workflows/nodejs.yml b/.github/workflows/nodejs.yml new file mode 100644 index 0000000..ee8cfa0 --- /dev/null +++ b/.github/workflows/nodejs.yml @@ -0,0 +1,32 @@ +name: Node CI + +on: [push] + +jobs: + build: + + runs-on: ubuntu-latest + + strategy: + matrix: + node-version: [8.x, 10.x, 12.x] + + services: + redisgraph: + image: redislabs/redisearch:latest + ports: + - 6379:6379 + + steps: + - uses: actions/checkout@v1 + - name: Use Node.js ${{ matrix.node-version }} + uses: actions/setup-node@v1 + with: + node-version: ${{ matrix.node-version }} + - name: npm install, build, and test + run: | + npm ci + npm run build --if-present + npm test + env: + CI: true diff --git a/.gitignore b/.gitignore index e3c2b3d..9bdf619 100644 --- a/.gitignore +++ b/.gitignore @@ -60,3 +60,6 @@ typings/ .DS_Store *.sock testing.js + +# misc + diff --git a/History.md b/History.md index 957e659..ca9f812 100644 --- a/History.md +++ b/History.md @@ -1,74 +1,4 @@ -1.0.0 / +0.0.1 / July 2017 ================== + * Forked from Reds - * Updated tests and benchmarks to take a connection object from an command line argument - * Updated creaky dependencies - * Added the ability to plug in new natural language processor and options - -0.2.4 / 2013-08-10 -================== - - * add function to limit the number of results (pagination) - * fix failure on strings that do not contain words - -0.2.3 / 2013-02-22 -================== - - * fix redis call to allow for multiple keywords - -0.2.2 / 2012-10-08 -================== - - * update natural. Closes #16 - -0.2.1 / 2012-09-05 -================== - - * handle punctuation better [kbsymanz] - -0.2.0 / 2012-08-21 -================== - - * add weighted search [kbsymanz] - * update redis dep - * update natural dep - -0.1.4 / 2012-06-28 -================== - - * grr engines - -0.1.2 / 2012-01-30 -================== - - * Upgrade natural dep. Closes #7 - -0.1.1 / 2011-11-16 -================== - - * Added support for node v0.5.x and v0.6.x - * Added `Query` - -0.1.0 / 2011-07-28 -================== - - * Added `Search` to allow multiple searches namespaced within Redis - * Added _web-search_ example - * Added _web-index_ example - -0.0.3 / 2011-07-27 -================== - - * Added simple example - * Changed: default `reds.search()` to intersection - -0.0.2 / 2011-07-27 -================== - - * Added `make benchmark` - * Changed: generate only a single redis client - -0.0.1 / 2011-07-27 -================== - - * Initial release diff --git a/Readme.md b/Readme.md index b397940..d8824f1 100644 --- a/Readme.md +++ b/Readme.md @@ -1,22 +1,30 @@ -# reds +[![Actions Status](https://github.com/stockholmux/redredisearch/workflows/Node%20CI/badge.svg)](https://github.com/stockholmux/redredisearch/actions) +[![npm version](https://badge.fury.io/js/redredisearch.svg)](https://badge.fury.io/js/redredisearch) - reds is a light-weight Redis search for node.js. This module was originally developed to provide search capabilities for [Kue](http://automattic.github.io/kue/) a priority job queue, however it is very much a light general purpose search library that could be integrated into a blog, a documentation server, etc. +# RedRediSearch + RedRediSearch is a Node.js wrapper library for the [RediSearch](http://redisearch.io/) Redis module. It is more-or-less syntactically compatible with [Reds](https://github.com/tj/reds), another Node.js search library. RedRediSearch and RediSearch can provide full-text searching that is much faster than the original Reds library (see Benchmarks). + + ## Upgrading -Version 1.0.0 is syntactically compatible with previous versions of reds (0.2.5). However, [natural](https://github.com/NaturalNode/natural) has been updated. Documents indexed with older installs of reds (using natural v0.2.0) may need to be re-indexed to avoid some edge cases. +If you are upgrading from Reds, you'll need to make your `createSearch` asynchronous and re-index your data. Otherwise, your app-level logic and code should be compatible. ## Installation - $ npm install reds + $ npm install redredisearch ## Example -The first thing you'll want to do is create a `Search` instance, which allows you to pass a `key`, used for namespacing within Redis so that you may have several searches in the same db. You may specify your own [node_redis](https://github.com/NodeRedis/node_redis) instance with the `reds.setClient` function. +The first thing you'll want to do is create a `Search` instance, which allows you to pass a `key`, used for namespacing within RediSearch so that you may have several searches in the same Redis database. You may specify your own [node_redis](https://github.com/NodeRedis/node_redis) instance with the `redredisearch.setClient` function. - var search = reds.createSearch('pets'); +```js +redredisearch.createSearch('pets',{}, function(err, search) { + /* ... */ +}); +``` - reds acts against arbitrary numeric or string based ids, so you could utilize this library with essentially anything you wish, even combining data stores. The following example just uses an array for our "database", containing some strings, which we add to reds by calling `Search#index()` padding the body of text and an id of some kind, in this case the index. +You can then add items to the index with the `Search#index` function. ```js var strs = []; @@ -28,36 +36,37 @@ strs.push('Manny is a cat'); strs.push('Luna is a cat'); strs.push('Mustachio is a cat'); -strs.forEach(function(str, i){ search.index(str, i); }); +redredisearch.createSearch('pets',{}, function(err,search) { + strs.forEach(function(str, i){ search.index(str, i); }); +}); ``` - To perform a query against reds simply invoke `Search#query()` with a string, and pass a callback, which receives an array of ids when present, or an empty array otherwise. + To perform a query against the index simply invoke `Search#query()` with a string, and pass a callback, which receives an array of ids when present, or an empty array otherwise. ```js search - .query(query = 'Tobi dollars') + .query('Tobi dollars') .end(function(err, ids){ if (err) throw err; console.log('Search results for "%s":', query); ids.forEach(function(id){ console.log(' - %s', strs[id]); }); - process.exit(); }); ``` - By default reds performs an intersection of the search words. The previous example would yield the following output since only one string contains both "Tobi" _and_ "dollars": + By default, queries are an intersection of the search words. The previous example would yield the following output since only one string contains both "Tobi" _and_ "dollars": ``` Search results for "Tobi dollars": - Tobi wants four dollars ``` - We can tweak reds to perform a union by passing either "union" or "or" to `Search#type()` in `reds.search()` between `Search#query()` and `Search#end()`, indicating that _any_ of the constants computed may be present for the id to match. + We can tweak the query to perform a union by passing either "union" or "or" to `Search#type()` in `redredisearch.search()` between `Search#query()` and `Search#end()`, indicating that _any_ of the constants computed may be present for the `id` to match. ```js search - .query(query = 'tobi dollars') + .query('tobi dollars') .type('or') .end(function(err, ids){ if (err) throw err; @@ -65,7 +74,6 @@ search ids.forEach(function(id){ console.log(' - %s', strs[id]); }); - process.exit(); }); ``` @@ -78,79 +86,118 @@ Search results for "tobi dollars": - Loki, Jane, and Tobi are ferrets ``` +RediSearch has an advanced query syntax that can be used by using the 'direct' search type. See the [RediSearch documentation](http://redisearch.io/Query_Syntax/) for this syntax. + +```js +search + .query('(hello|hella) (world|werld)') + .type('direct') + .end(function(err, ids){ + /* ... */ + }); +``` + +Also included in the package is the RediSearch Suggestion API. This has no corollary in the Reds module. The Suggestion API is ideal for auto-complete type situations and is entirely separate from the Search API. + +```js +var suggestions = redredisearch.suggestion('my-suggestion-list'); + +suggestions.add( + 'redis', // add 'redis' + 2, // with a 'score' of 2, this affects the position in the results, higher = higher up in results + function(err,sizeOfSuggestionList) { /* ... */ } // callback +); +suggestions.add( + 'redisearch', + 5, + function(err,sizeOfSuggestionList) { /* ... */ } +); +suggestions.add( + 'reds', + 1, + function(err,sizeOfSuggestionList) { /* ... */ } +); + +/* ... */ + +sugggestions.get( + 're', // prefix - will find anything starting with "re" + function(err, returnedSuggestions) { + /* returnedSuggestions is set to [ "redisearch", "redis", "reds" ] */ + } +); + +sugggestions.get( + 'redis', // prefix - will find anything starting with "redis", so not "reds" + function(err, returnedSuggestions) { + /* returnedSuggestions is set to [ "redisearch", "redis" ] */ + } +) +``` + +There is also a `fuzzy` opt and `maxResults` that can either be set by chaining or by passing an object in the second argument in the constructor. + + ## API ```js -reds.createSearch(key) +redredisearch.createSearch(key, options, fn) : Search +redredisearch.setClient(inClient) +redredisearch.createClient() +redredisearch.confirmModule(cb) +redredisearch.words(str) : Array +redredisearch.suggestionList(key,opts) : Suggestion Search#index(text, id[, fn]) Search#remove(id[, fn]); -Search#query(text, fn[, type]); +Search#query(text, fn[, type]) : Query +Query#type(type) +Query#between(str) +Query#end(fn) +Suggestion#fuzzy(isFuzzy) +Suggestion#maxResults(maxResults) +Suggestion#add(str,score,fn) +Suggestion#get(prefix,fn) +Suggestion#del(str,fn) + ``` Examples: ```js -var search = reds.createSearch('misc'); +var search = redredisearch.createSearch('misc'); search.index('Foo bar baz', 'abc'); search.index('Foo bar', 'bcd'); search.remove('bcd'); search.query('foo bar').end(function(err, ids){}); ``` -## Extending reds - -Starting in 1.0.0, you can easily extend and expand how reds functions. When creating a new search, supply an object as the second argument. There are currently three properties that can be configured: - -- `nlpProcess` the natural language processing function. You can alter how the words are processed (split, stemmed, and converted to metaphones) using this function. -- `writeIndex` how the items are written to the index. -- `removeIndex` how the items are removed from the index. - -See the `lib/reds.js` file for the implementation of each. Please keep in mind that changing these functions may invalidate your previously stored index. - -```js -reds.createSearch('pets', { - nlpProcess : yourNlpProcessingFunction, - writeIndex : yourWriteIndexFunction, - removeIndex : yourRemoveIndexFunction -}); -``` -## About - - Currently reds strips stop words and applies the metaphone and porter stemmer algorithms to the remaining words before mapping the constants in Redis sets. For example the following text: - - Tobi is a ferret and he only wants four dollars +## Benchmarks - Converts to the following constant map: - -```js -{ - Tobi: 'TB', - ferret: 'FRT', - wants: 'WNTS', - four: 'FR', - dollars: 'DLRS' -} -``` +When compared to Reds, RedRediSearch is much faster at indexing and somewhat faster at query: - This also means that phonetically similar words will match, for example "stefen", "stephen", "steven" and "stefan" all resolve to the constant "STFN". Reds takes this further and applies the porter stemming algorithm to "stem" words, for example "counts", and "counting" become "count". +_Indexing - documents / second_ - Consider we have the following bodies of text: +| Module | Tiny | Small | Medium | Large | +|----------------|------|-------|--------|-------| +| Reds | 122 | 75 | 10 | 0 | +| RediRediSearch | 1,256| 501 | 132 | 5 | - Tobi really wants four dollars - For some reason tobi is always wanting four dollars +_Query - queries / second_ - The following search query will then match _both_ of these bodies, and "wanting", and "wants" both reduce to "want". +| Module | 1 term | 2 terms / AND | 2 terms / OR | 3 terms / AND | 3 terms / OR | Long* / AND | Long* / OR | +|----------------|--------|---------------|--------------|---------------|--------------|------------|----------| +| Reds | 8,754 | 8,765 | 8,389 | 7,622 | 7,193 | 1,649 | 1,647 | +| RedRediSearch | 10,955 | 12,945 | 10,054 | 12,769 | 8,389 | 6,456 | 12,311 | - tobi wants four dollars +The "Long" query string is taken from the Canadian Charter of Rights and Freedoms: "Everyone has the following fundamental freedoms: (a) freedom of conscience and religion; (b) freedom of thought, belief, opinion and expression, including freedom of the press and other media of communication; (c) freedom of peaceful assembly; and (d) freedom of association." (Used because I just had it open in another tab...) -## Benchmarks +## Next steps - Nothing scientific but preliminary benchmarks show that a small 1.6kb body of text is currently indexed in ~__6ms__, or __163__ ops/s. Medium bodies such as 40kb operate around __6__ ops/s, or __166ms__. +- More coverage of RediSearch features +- Tests +- Better examples - Querying with a multi-word phrase, and an index containing ~3500 words operates around __5300__ ops/s. Not too bad. - - If working with massive documents, you may want to consider adding a "keywords" field, and simply indexing it's value instead of multi-megabyte documents. ## License @@ -158,6 +205,8 @@ reds.createSearch('pets', { Copyright (c) 2011 TJ Holowaychuk <tj@vision-media.ca> +Modified work Copyright (c) 2017 Kyle Davis + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including diff --git a/benchmarks/benchmark.reds.node.js b/benchmarks/benchmark.reds.node.js new file mode 100644 index 0000000..4ead9f8 --- /dev/null +++ b/benchmarks/benchmark.reds.node.js @@ -0,0 +1,90 @@ +// this will benchmark reds. The other benchmark (benchmark/index.js) will actually benchmark the package. This is included for comparison purposes. +// to run this you'll need to install the reds module with npm - not included becuase it's not really a dependency +var argv = require('yargs') + .demand('connection') + .argv; +var redis = require('redis'); +var connectionObj = require(argv.connection); +var reds; +var fs = require('fs'); + + +reds = require('reds'); + +reds.setClient(redis.createClient(connectionObj)); + +reds = reds.createSearch('reds'); +// test data + +var tiny = fs.readFileSync('./node_modules/reds/package.json', 'utf8'); +tiny = Array(5).join(tiny); +var small = fs.readFileSync('./node_modules/reds/Readme.md', 'utf8'); +var medium = Array(10).join(small); +var large = Array(30).join(medium); + +// benchmarks + +suite('indexing', function(){ + bench('tiny index', function(done){ + reds.index(tiny, 'reds1234', done); + }); + + bench('small index', function(done){ + reds.index(small, 'reds1234', done); + }); + + bench('medium index', function(done){ + reds.index(medium, 'reds1234', done); + }); + + bench('large', function(done){ + reds.index(large, 'reds1234', done); + }); + + bench('query - one term', function(done){ + reds + .query('one') + .end(done); + }); + + bench('query - two terms (and)', function(done){ + reds + .query('one two') + .end(done); + }); + + bench('query - two terms (or)', function(done){ + reds + .query('one two') + .type('or') + .end(done); + }); + + bench('query - three terms (and)', function(done){ + reds + .query('one two three') + .end(done); + }); + + bench('query - three terms (or)', function(done){ + reds + .query('one two three') + .type('or') + .end(done); + }); + + let rightsAndFreedoms = 'Everyone has the following fundamental freedoms: (a) freedom of conscience and religion; (b) freedom of thought, belief, opinion and expression, including freedom of the press and other media of communication; (c) freedom of peaceful assembly; and (d) freedom of association.'; + bench('query - long (and)', function(done){ + reds + .query(rightsAndFreedoms) + .end(done); + }); + + bench('query - long (or)', function(done){ + reds + .query(rightsAndFreedoms) + .type('or') + .end(done); + }); + +}); \ No newline at end of file diff --git a/benchmarks/index.js b/benchmarks/index.js index 91c90d5..601d756 100644 --- a/benchmarks/index.js +++ b/benchmarks/index.js @@ -1,45 +1,111 @@ - -/** - * Module dependencies. - */ +// this will benchmark redredisearch. Also included is the comparision benchmark for reds (benchmarks/benchmark.reds.node.js) as to compare the two packages var argv = require('yargs') - .demand('connection') + .demand('connection') // you need to provide the --connection in the cmd line arguments, it's a path to a JSON file of node_redis connection information .argv; var redis = require('redis'); var connectionObj = require(argv.connection); var reds; +var client = redis.createClient(connectionObj); var fs = require('fs'); reds = require('../'); -reds.setClient(redis.createClient(connectionObj)); +reds.setClient(client); -reds = reds.createSearch('reds'); // test data -var tiny = fs.readFileSync('package.json', 'utf8'); +var tiny = fs.readFileSync('../package.json', 'utf8'); tiny = Array(5).join(tiny); -var small = fs.readFileSync('Readme.md', 'utf8'); +var small = fs.readFileSync('../Readme.md', 'utf8'); var medium = Array(10).join(small); var large = Array(30).join(medium); -// benchmarks suite('indexing', function(){ - bench('tiny', function(done){ - reds.index(tiny, '1234', done); + var + search; + + before(function(next) { + reds.createSearch('redisearch', {}, function(err,redisearch) { + if (err) { throw err; } + search = redisearch; + next(); + }); + }); + + bench('tiny index', function(done){ + search.index(tiny, 'redisearch1234', done); }); - bench('small', function(done){ - reds.index(small, '1234', done); + bench('small index', function(done){ + search.index(small, 'redisearch1234', done); }); - bench('medium', function(done){ - reds.index(medium, '1234', done); + bench('medium index', function(done){ + search.index(medium, 'redisearch1234', done); + }); + + bench('large index', function(done){ + search.index(large, 'redisearch1234', done); + }); + + bench('query - one term', function(done){ + search + .query('one') + .end(done); }); - bench('large', function(done){ - reds.index(large, '1234', done); + bench('query - two terms (and)', function(done){ + search + .query('one two') + .end(done); }); -}); \ No newline at end of file + + bench('query - two terms (or)', function(done){ + search + .query('one two') + .type('or') + .end(done); + }); + + bench('query - three terms (and)', function(done){ + search + .query('one two three') + .end(done); + }); + + bench('query - three terms (or)', function(done){ + search + .query('one two three') + .type('or') + .end(done); + }); + + let rightsAndFreedoms = 'Everyone has the following fundamental freedoms: (a) freedom of conscience and religion; (b) freedom of thought, belief, opinion and expression, including freedom of the press and other media of communication; (c) freedom of peaceful assembly; and (d) freedom of association.'; + bench('query - long (and)', function(done){ + search + .query(rightsAndFreedoms) + .end(done); + }); + + bench('query - long (or)', function(done){ + search + .query(rightsAndFreedoms) + .type('or') + .end(done); + }); + + bench('query - direct / complex', function(done){ + search + .query('(dog|cat) (lassie|garfield)') + .type('direct') + .end(done); + }); + + after(function() { + client.quit(); + }); +}); + + diff --git a/examples/simple.js b/examples/simple.js index aaed810..53e216f 100644 --- a/examples/simple.js +++ b/examples/simple.js @@ -1,44 +1,47 @@ +const + argv = require('yargs') // command line handling + .demand('connection') // require the 'connection' argument + .demand('query') // the query we'll run against the indexed values + .argv, + redsearch = require('../'), // RedRediSearch, syntax compatible with Reds + redis = require('redis'), // node_redis module + creds = require(argv.connection), // load the JSON specified in the argument + client = redis.createClient(creds); // create a Redis client with the Node_redis connection object + +redsearch.setClient(client); // associate the correct client. + +redsearch.createSearch('pets', {}, function(err,search) { + // $ node examples/simple --connection /path/to/connection/object/json --query tobi + // $ node examples/simple --connection /path/to/connection/object/json --query tobi + // $ node examples/simple --connection /path/to/connection/object/json --query cat + // $ node examples/simple --connection /path/to/connection/object/json --query fun + // $ node examples/simple --connection /path/to/connection/object/json --query "funny ferret" + + var strs = []; + strs.push('Manny is a cat'); + strs.push('Luna is a cat'); + strs.push('Tobi is a ferret'); + strs.push('Loki is a ferret'); + strs.push('Jane is a ferret'); + strs.push('Jane is funny ferret'); + + // index them + + strs.forEach(function(str, i){ + search.index(str, i); + }); -/** - * Module dependencies. - */ - -var reds = require('../') - , search = reds.createSearch('pets'); - -// $ node examples/simple Tobi -// $ node examples/simple tobi -// $ node examples/simple cat -// $ node examples/simple bitch -// $ node examples/simple bitch ferret - -var strs = []; -strs.push('Manny is a cat'); -strs.push('Luna is a cat'); -strs.push('Tobi is a ferret'); -strs.push('Loki is a ferret'); -strs.push('Jane is a ferret'); -strs.push('Jane is bitchy ferret'); - -var query = process.argv.slice(2).join(' '); -if (!query) throw new Error('query required'); - -// index them - -strs.forEach(function(str, i){ - search.index(str, i); -}); - -// query - -search.query(query).end(function(err, ids){ - if (err) throw err; - var res = ids.map(function(i){ return strs[i]; }); - console.log(); - console.log(' Search results for "%s"', query); - res.forEach(function(str){ - console.log(' - %s', str); + // query + + search.query(argv.query).end(function(err, ids){ + if (err) throw err; + var res = ids.map(function(i){ return strs[i]; }); + console.log(); + console.log(' Search results for "%s"', argv.query); + res.forEach(function(str){ + console.log(' - %s', str); + }); + console.log(); + process.exit(); }); - console.log(); - process.exit(); -}); \ No newline at end of file +}); diff --git a/examples/form.html b/examples/static/form.html similarity index 90% rename from examples/form.html rename to examples/static/form.html index 8cadbca..c555ddd 100644 --- a/examples/form.html +++ b/examples/static/form.html @@ -35,7 +35,7 @@ var val = o(this).val().trim() , results = o('#results').empty(); if (!val) return; - o.get('/search?q=' + val, function(res){ + o.getJSON('/search?q=' + val, function(res){ res.forEach(function(res){ results.append('
  • ' + res + '
    • '); }); @@ -47,7 +47,7 @@
    -

    Try "express", "mongodb", "hacker" ...

    +

    Try "express", "redis", "hacker" ...

    diff --git a/examples/urls b/examples/urls deleted file mode 100644 index 38413d4..0000000 --- a/examples/urls +++ /dev/null @@ -1,20 +0,0 @@ -http://learnboost.com -http://nodejs.org -http://expressjs.com -http://expressjs.com/guide.html -http://expressjs.com/applications.html -http://jade-lang.com -http://google.com -http://ign.com -http://1up.com -http://gamespot.com -http://yahoo.com -http://purevolume.com -http://icefilms.info -http://mongoosejs.com/ -http://mongoosejs.com/docs/model-definition.html -http://mongoosejs.com/docs/embedded-documents.html -http://mongoosejs.com/docs/middleware.html -http://mongoosejs.com/docs/indexes.html -http://mongoosejs.com/docs/virtuals.html -http://news.ycombinator.com \ No newline at end of file diff --git a/examples/urls.json b/examples/urls.json new file mode 100644 index 0000000..d587e24 --- /dev/null +++ b/examples/urls.json @@ -0,0 +1,20 @@ +[ + "http://learnboost.com", + "http://nodejs.org", + "http://expressjs.com", + "http://expressjs.com/guide.html", + "http://expressjs.com/applications.html", + "http://pugjs.com", + "http://google.com", + "http://ign.com", + "http://1up.com", + "http://gamespot.com", + "http://yahoo.com", + "http://purevolume.com", + "http://icefilms.info", + "http://mongoosejs.com/", + "http://mongoosejs.com/docs/api.html", + "http://news.ycombinator.com", + "http://redis.io", + "http://redislabs.com" +] \ No newline at end of file diff --git a/examples/web-index.js b/examples/web-index.js index 5c2e9e8..ba744f5 100644 --- a/examples/web-index.js +++ b/examples/web-index.js @@ -1,59 +1,51 @@ +const + argv = require('yargs') // command line handling + .demand('connection') // require the 'connection' argument (this is a node_redis connection object in JSON Format) + .argv, + redsearch = require('../'), // RedRediSearch, syntax compatible with Reds + redis = require('redis'), // node_redis module + request = require('request'), // Get a remote URL + creds = require(argv.connection), // load the JSON specified in the argument + client = redis.createClient(creds), // create a Redis client with the Node_redis connection object + urls = require('./urls.json'), // load the URLs from a JSON file + start = new Date; // for calculating time of index + +function striptags(html) { // quick and dirty, don't reuse ("Lame" according to TJ) + return String(html).replace(/<\/?([^>]+)>/g, ''); +} -/** - * Module dependencies. - */ - -var reds = require('../') - , agent = require('superagent') - , search = reds.createSearch('webpages') - , fs = require('fs'); - -// install local dev deps first: -// $ npm install -d -// then run -// $ node examples/web-index - -var urls = fs.readFileSync(__dirname + '/urls', 'utf8').split('\n') - , start = new Date; - -// index - -var pending = urls.length; - -urls.forEach(function(url, i){ - function log(msg) { - console.log(' \033[90m%s \033[36m%s\033[0m', msg, url); - } - - log('fetching'); - agent.get(url, function(res){ - var words; - - // strip html tags - log('stripping tags'); - words = striptags(res.text); - - // index - log('indexing'); - search.index(words, i, function(err){ - if (err) throw err; - log('completed'); - --pending || done(); +redsearch.setClient(client); // associate the correct client. + +redsearch.createSearch('web',{},function(err,search) { // create the search with at the "web" key + var pending = urls.length; + + urls.forEach(function(url, i){ // over each URL + function log(msg) { // logging for this specific URL + console.log( + ' \033[90m%s \033[36m%s\033[0m', msg, url + ); + } + log('fetching'); + + request(url, function(err, res, body){ + if (err) throw err; // error 'handling' + var words = striptags(body); // strip html tags + + log('indexing'); + search.index(words, i, function(err){ // words are being indexed and the ID is just a number here + if (err) throw err; + log('completed'); + --pending || done(); // if pending drops to 0 then call done. + }); + }); }); - }); -}); -// all done - -function done() { - console.log(' indexed %d pages in %ds' - , urls.length - , ((new Date - start) / 1000).toFixed(2)); - process.exit(); -} + // all done -// lame, dont use me - -function striptags(html) { - return String(html).replace(/<\/?([^>]+)>/g, ''); -} + function done() { // wrap up + console.log(' indexed %d pages in %ds' + , urls.length + , ((new Date - start) / 1000).toFixed(2)); + client.quit(); + } +}); \ No newline at end of file diff --git a/examples/web-realtime.js b/examples/web-realtime.js deleted file mode 100644 index 9d5b7a3..0000000 --- a/examples/web-realtime.js +++ /dev/null @@ -1,40 +0,0 @@ - -/** - * Module dependencies. - */ - -var http = require('http') - , reds = require('../') - , search = reds.createSearch('webpages') - , parse = require('url').parse - , qs = require('querystring') - , fs = require('fs'); - -// urls, could be in redis or another db - -var urls = fs.readFileSync(__dirname + '/urls', 'utf8').split('\n'); - -// First run: -// $ node examples/web-index - -http.createServer(function(req, res){ - var url = parse(req.url) - , query = qs.parse(url.query); - - if ('/search' == url.pathname) { - search.query(query.q).end(function(err, ids){ - // ids are simply indexes in this case - var json = JSON.stringify(ids.map(function(id){ return urls[id]; })); - res.setHeader('Content-Type', 'application/json'); - res.setHeader('Content-Length', json.length); - res.end(json); - }); - } else { - res.setHeader('Content-Type', 'text/html'); - fs.readFile(__dirname + '/form.html', 'utf8', function(err, buf){ - res.end(buf); - }); - } -}).listen(3000); - -console.log('App started on port 3000'); \ No newline at end of file diff --git a/examples/web-search.js b/examples/web-search.js deleted file mode 100644 index 8bb0fca..0000000 --- a/examples/web-search.js +++ /dev/null @@ -1,35 +0,0 @@ - -/** - * Module dependencies. - */ - -var reds = require('../') - , fs = require('fs') - , search = reds.createSearch('webpages') - , urls = fs.readFileSync(__dirname + '/urls', 'utf8').split('\n'); - -// First run: -// $ node examples/web-index -// Then query: -// $ node examples/web-search whatever query here -// $ node examples/web-search jade -// $ node examples/web-search education -// $ node examples/web-search learnboost -// $ node examples/web-search mongodb - -var query = process.argv.slice(2).join(' '); -if (!query) throw new Error('query required'); - -// query - -search.query(query).end(function(err, ids){ - if (err) throw err; - var res = ids.map(function(i){ return urls[i]; }); - console.log(); - console.log(' Search results for "%s"', query); - res.forEach(function(str){ - console.log(' - %s', str); - }); - console.log(); - process.exit(); -}); \ No newline at end of file diff --git a/examples/web.demo.node.js b/examples/web.demo.node.js new file mode 100644 index 0000000..2d57169 --- /dev/null +++ b/examples/web.demo.node.js @@ -0,0 +1,37 @@ +const + argv = require('yargs') // command line handling + .demand('connection') // require the 'connection' argument (this is a node_redis connection object in JSON Format) + .argv, + redsearch = require('../'), // RedRediSearch, syntax compatible with Reds + redis = require('redis'), // node_redis module + creds = require(argv.connection), // load the JSON specified in the argument + client = redis.createClient(creds), // create a Redis client with the Node_redis connection object + express = require('express'), // simple web server module + urls = require('./urls.json'), // load the URLs from a JSON file + app = express(), // server instance + port = 3000; // load demo on http://localhost:3000/ + +redsearch.setClient(client); // associate the correct client. + +redsearch.createSearch('web',{},function(err,search) { // create the search with at the "web" key + app.get( // HTTP Get + '/search', // route for /search + function(req,res,next) { + search + .query(req.query.q) // /search?q=[search query] + .end(function(err, ids){ + if (err) { next(err); } else { // error handling + res.json( // return JSON + ids.map(function(id){ return urls[id]; }) // this will return all the URLs that match the results + ); + } + }); + } + ); + + app + .use(express.static('static')) // server out static files (the form) + .listen(port,function() { // start at `port` + console.log('Listening at',port); // we're loaded - let the console know + }); +}); diff --git a/index.js b/index.js index ab43d11..ec4c9af 100644 --- a/index.js +++ b/index.js @@ -1,2 +1,2 @@ -module.exports = require('./lib/reds'); \ No newline at end of file +module.exports = require('./lib/redredisearch.js'); \ No newline at end of file diff --git a/lib/redredisearch.js b/lib/redredisearch.js new file mode 100644 index 0000000..798a6b0 --- /dev/null +++ b/lib/redredisearch.js @@ -0,0 +1,460 @@ +/*! + * redredisearch + * + * Forked from tj/reds + * Original work Copyright(c) 2011 TJ Holowaychuk + * Modified work Copyright(c) 2017 Kyle Davis + * MIT Licensed + */ + +/** + * Module dependencies. + */ + + +var redis = require('redis'); +function noop(){}; + +/** + * Library version. + */ + +exports.version = '0.0.1'; + +/** + * Expose `Search`. + */ + +exports.Search = Search; + +/** + * Expose `Query`. + */ + +exports.Query = Query; + +/** + * Search types. + */ + +var types = { + intersect: 'and', + union: 'or', + and: 'and', + or: 'or' +}; + +/** + * Alternate way to set client + * provide your own behaviour. + * + * @param {RedisClient} inClient + * @return {RedisClient} + * @api public + */ + +exports.setClient = function(inClient) { + return exports.client = inClient; +} + +/** + * Create a redis client, override to + * provide your own behaviour. + * + * @return {RedisClient} + * @api public + */ + +exports.createClient = function(){ + return exports.client + || (exports.client = redis.createClient()); +}; + +/** + * Confirm the existence of the RediSearch Redis module + * + * @api public + */ + +exports.confirmModule = function(cb) { + exports.client.send_command('ft.create',[], function(err) { + let strMsg = String(err); + if (strMsg.indexOf('ERR wrong number of arguments') > 0) { + cb(null); + } else { + cb(err); + } + }); +} + +/** + * Return a new reds `Search` with the given `key`. + * @param {String} key + * @param {Object} opts + * @return {Search} + * @api public + */ + +exports.createSearch = function(key,opts,cb){ + const + searchObj = function(err,info) { + if (err) { cb(err); } else { + cb(err,new Search(key,info,opts)); + } + }; + + opts = !opts ? {} : opts; + opts.payloadField = opts.payloadField ? opts.payloadField : 'payload'; + + if (!key) throw new Error('createSearch() requires a redis key for namespacing'); + + exports.client.send_command('FT.INFO',[key],function(err,info) { + if (err) { + //if the index is not found, we need to make it. + if (String(err).indexOf('Unknown Index name') > 0 ){ + let args = [ + key, + 'SCHEMA', opts.payloadField, 'text' + ]; + exports.client.send_command( + 'FT.CREATE', + args, + function(err) { + if (err) { cb(err); } else { + exports.client.send_command('FT.INFO',[key],searchObj); + } + } + ); + } + + } else { searchObj(err,info); } + }); +}; + +/** + * Return the words in `str`. This is for compatability reasons (convert OR queries to pipes) + * + * @param {String} str + * @return {Array} + * @api private + */ + +exports.words = function(str){ + return String(str).match(/\w+/g); +}; + + +/** + * Initialize a new `Query` with the given `str` + * and `search` instance. + * + * @param {String} str + * @param {Search} search + * @api public + */ + +function Query(str, search) { + this.str = str; + this.type('and'); + this.search = search; +} + +/** + * Set `type` to "union" or "intersect", aliased as + * "or" and "and". + * + * @param {String} type + * @return {Query} for chaining + * @api public + */ + +Query.prototype.type = function(type){ + if (type === 'direct') { + this._directQuery = true; + } else { + this._direct = false; + this._type = types[type]; + } + return this; +}; + +/** + * Limit search to the specified range of elements. + * + * @param {String} start + * @param {String} stop + * @return {Query} for chaining + * @api public + */ +Query.prototype.between = function(start, stop){ + this._start = start; + this._stop = stop; + return this; +}; + +/** + * Perform the query and callback `fn(err, ids)`. + * + * @param {Function} fn + * @return {Query} for chaining + * @api public + */ + +Query.prototype.end = function(fn){ + var + key = this.search.key, + db = this.search.client, + query = this.str, + direct = this._directQuery, + args = [], + joiner = ' ', + rediSearchQuery; + + if (direct) { + rediSearchQuery = query; + } else { + rediSearchQuery = exports.words(query); + if (this._type === 'or') { + joiner = '|' + } + rediSearchQuery = rediSearchQuery.join(joiner); + } + args = [ + key, + rediSearchQuery, + 'NOCONTENT' + ]; + if (this._start !== undefined) { + args.push('LIMIT',this._start,this._stop); + } + + db.send_command( + 'FT.SEARCH', + args, + function(err,resp) { + if (err) { fn(err); } else { + fn(err,resp.slice(1)); + } + } + ); + + return this; +}; + +/** + * Initialize a new `Suggestion` with the given `key`. + * + * @param {String} key + * @param {Object} opts + * @api public + */ +var Suggestion = function(key,opts) { + this.key = key; + this.client = exports.createClient(); + this.opts = opts || {}; + if (this.opts.fuzzy) { + this.fuzzy = opts.fuzzy; + } + if (this.opts.maxResults) { + this.maxResults = opts.maxResults; + } + if (this.opts.incr) { + this.incr = opts.incr; + } + if (this.opts.withPayloads) { + this.withPayloads = true; + } +} + +/** + * Create a new Suggestion object + * + * @param {String} key + * @param {Object} opts + * @api public + */ +exports.suggestionList = function(key,opts) { + return new Suggestion(key,opts); +} + +/** + * Set `fuzzy` on suggestion get. Can also be set via opts in the constructor + * + * @param {Boolean} isFuzzy + * @return {Suggestion} for chaining + * @api public + */ + +Suggestion.prototype.fuzzy = function(isFuzzy){ + this.fuzzy = isFuzzy; + return this; +}; + +/** + * Set the max number of returned suggestions. Can also be set via opts in the constructor + * + * @param {Number} maxResults + * @return {Suggestion} for chaining + * @api public + */ + +Suggestion.prototype.maxResults = function(maxResults){ + this.maxResults = maxResults; + return this; +}; + +Suggestion.prototype.add = function(str,score,payload,fn) { + if((typeof fn === 'undefined' || fn === null) && typeof payload === "function"){ + if(typeof fn !== 'undefined'){ + fn = payload; + } else { + var fn = payload; + } + payload = null; + }; + + var key = this.key; + var db = this.client; + var args = [ + key, + str, + score, + ]; + if (this.incr) { + args.push('INCR'); + } + if(payload !== null){ + args.push('PAYLOAD', (typeof payload === 'object' ? JSON.stringify(payload) : payload.toString())); + } + db.send_command( + 'FT.SUGADD', + args, + fn || noop + ); + return this; +} + +Suggestion.prototype.get = function(prefix,fn) { + var key = this.key; + var db = this.client; + var args = [ + key, + prefix + ]; + if (this.fuzzy) { + args.push('FUZZY'); + } + if (this.maxResults) { + args.push('MAX',this.maxResults); + } + if (this.withPayloads) { + args.push('WITHPAYLOADS'); + } + + db.send_command( + 'FT.SUGGET', + args, + fn + ); + + return this; +} + +Suggestion.prototype.del = function(str,fn) { + var key = this.key; + var db = this.client; + + db.send_command( + 'FT.SUGDEL', + [ + key, + str + ], + fn + ); + + return this; +} + +/** + * Initialize a new `Search` with the given `key`. + * + * @param {String} key + * @api public + */ + +function Search(key,info,opts) { + this.key = key; + this.client = exports.createClient(); + this.opts = opts; +} + +/** + * Index the given `str` mapped to `id`. + * + * @param {String} str + * @param {Number|String} id + * @param {Function} fn + * @api public + */ + +Search.prototype.index = function(str, id, fn){ + var key = this.key; + var db = this.client; + var opts = this.opts; + + db.send_command( + 'FT.ADD', + [ + key, + id, + 1, //default - this should be to be set in future versions + 'NOSAVE', //emulating Reds original behaviour + 'REPLACE', //emulating Reds original behaviour + 'FIELDS', + opts.payloadField, + str + ], + fn || noop + ); + + return this; +}; + +/** + * Remove occurrences of `id` from the index. + * + * @param {Number|String} id + * @api public + */ + +Search.prototype.remove = function(id, fn){ + fn = fn || noop; + var key = this.key; + var db = this.client; + + //this.removeIndex(db, id, key, fn); + db.send_command( + 'FT.DEL', + [ + key, + id + ], + fn + ) + + return this; +}; + +/** + * Perform a search on the given `query` returning + * a `Query` instance. + * + * @param {String} query + * @param {Query} + * @api public + */ + +Search.prototype.query = function(query){ + return new Query(query, this); +}; diff --git a/lib/reds.js b/lib/reds.js deleted file mode 100644 index aa62c0d..0000000 --- a/lib/reds.js +++ /dev/null @@ -1,430 +0,0 @@ -/*! - * reds - * Copyright(c) 2011 TJ Holowaychuk - * MIT Licensed - */ - -/** - * Module dependencies. - */ - -var natural = require('natural'); -var metaphone = natural.Metaphone.process; -var stem = natural.PorterStemmer.stem; -var stopwords = natural.stopwords; -var redis = require('redis'); -function noop(){}; - -/** - * Library version. - */ - -exports.version = '1.0.0'; - -/** - * Expose `Search`. - */ - -exports.Search = Search; - -/** - * Expose `Query`. - */ - -exports.Query = Query; - -/** - * Search types. - */ - -var types = { - intersect: 'zinterstore', - union: 'zunionstore', - and: 'zinterstore', - or: 'zunionstore' -}; - -/** - * Alternate way to set client - * provide your own behaviour. - * - * @param {RedisClient} inClient - * @return {RedisClient} - * @api public - */ - -exports.setClient = function(inClient) { - return exports.client = inClient; -} - -/** - * Create a redis client, override to - * provide your own behaviour. - * - * @return {RedisClient} - * @api public - */ - -exports.createClient = function(){ - return exports.client - || (exports.client = redis.createClient()); -}; - -/** - * Process indexing string into NLP components (words, counts, map, keys) - * - * @param {String} str - * @param {String} [key] - */ -exports.nlpProcess = function(str, key) { - var words = exports.stem(exports.stripStopWords(exports.words(str))); - var counts = exports.countWords(words); - var map = exports.metaphoneMap(words); - var keys = Object.keys(map); - //the key argument is only needed for removing items from the index - //by not executing the `metaphoneKeys` function, it speeds up indexing a bit - var metaphoneKeys = !key ? null : exports.metaphoneKeys(key, words) - - - return { - words : words, - counts : counts, - map : map, - keys : keys, - metaphoneKeys - : metaphoneKeys - }; -} - -/** - * Writes index to Redis - * - * @param {Object} db Redis Client - * @param {Number|String} id - * @param {String} key - * @param {Object} nlpObj Object with word map, counts, and keys - * @param {Function} fn - * - */ -exports.writeIndex = function(db, id, key, nlpObj, fn) { - var cmds = []; - nlpObj.keys.forEach(function(word, i){ - cmds.push(['zadd', key + ':word:' + nlpObj.map[word], nlpObj.counts[word], id]); - cmds.push(['zadd', key + ':object:' + id, nlpObj.counts[word], nlpObj.map[word]]); - }); - db.multi(cmds).exec(fn || noop); -} - -/** - * Removes index from Redis - * - * @param {Object} db Redis Client - * @param {Number|String} id - * @param {String} key - * @param {Function} fn - * - */ -exports.removeIndex = function(db, id, key, fn) { - db.zrevrangebyscore(key + ':object:' + id, '+inf', 0, function(err, constants){ - if (err) return fn(err); - var multi = db.multi().del(key + ':object:' + id); - constants.forEach(function(c){ - multi.zrem(key + ':word:' + c, id); - }); - multi.exec(fn); - }); -} - - -/** - * Return a new reds `Search` with the given `key`. - * @param {Object} opts - * @param {String} key - * @return {Search} - * @api public - */ - -exports.createSearch = function(key,opts){ - if (!key) throw new Error('createSearch() requires a redis key for namespacing'); - - opts = !opts ? {} : opts; - opts.nlpProcess = !opts.nlpProcess ? exports.nlpProcess : opts.nlpProcess; - opts.writeIndex = !opts.writeIndex ? exports.writeIndex : opts.writeIndex; - opts.removeIndex = !opts.removeIndex ? exports.removeIndex : opts.removeIndex; - - return new Search(key, opts.nlpProcess, opts.writeIndex, opts.removeIndex); -}; - -/** - * Return the words in `str`. - * - * @param {String} str - * @return {Array} - * @api private - */ - -exports.words = function(str){ - return String(str).match(/\w+/g); -}; - -/** - * Stem the given `words`. - * - * @param {Array} words - * @return {Array} - * @api private - */ - -exports.stem = function(words){ - var ret = []; - if (!words) return ret; - for (var i = 0, len = words.length; i < len; ++i) { - ret.push(stem(words[i])); - } - return ret; -}; - -/** - * Strip stop words in `words`. - * - * @param {Array} words - * @return {Array} - * @api private - */ - -exports.stripStopWords = function(words){ - var ret = []; - if (!words) return ret; - for (var i = 0, len = words.length; i < len; ++i) { - if (~stopwords.indexOf(words[i])) continue; - ret.push(words[i]); - } - return ret; -}; - -/** - * Returns an object mapping each word in a Array - * to the number of times it occurs in the Array. - * - * @param {Array} words - * @return {Object} - * @api private - */ - -exports.countWords = function(words){ - var obj = {}; - if (!words) return obj; - for (var i = 0, len = words.length; i < len; ++i) { - obj[words[i]] = (obj[words[i]] || 0) + 1; - } - return obj; -}; - -/** - * Return the given `words` mapped to the metaphone constant. - * - * Examples: - * - * metaphone(['tobi', 'wants', '4', 'dollars']) - * // => { '4': '4', tobi: 'TB', wants: 'WNTS', dollars: 'TLRS' } - * - * @param {Array} words - * @return {Object} - * @api private - */ - -exports.metaphoneMap = function(words){ - var obj = {}; - if (!words) return obj; - for (var i = 0, len = words.length; i < len; ++i) { - obj[words[i]] = metaphone(words[i]); - } - return obj; -}; - -/** - * Return an array of metaphone constants in `words`. - * - * Examples: - * - * metaphone(['tobi', 'wants', '4', 'dollars']) - * // => ['4', 'TB', 'WNTS', 'TLRS'] - * - * @param {Array} words - * @return {Array} - * @api private - */ - -exports.metaphoneArray = function(words){ - var arr = []; - var constant; - - if (!words) return arr; - - for (var i = 0, len = words.length; i < len; ++i) { - constant = metaphone(words[i]); - if (!~arr.indexOf(constant)) arr.push(constant); - } - - return arr; -}; - -/** - * Return a map of metaphone constant redis keys for `words` - * and the given `key`. - * - * @param {String} key - * @param {Array} words - * @return {Array} - * @api private - */ - -exports.metaphoneKeys = function(key, words){ - return exports.metaphoneArray(words).map(function(c){ - return key + ':word:' + c; - }); -}; - -/** - * Initialize a new `Query` with the given `str` - * and `search` instance. - * - * @param {String} str - * @param {Search} search - * @api public - */ - -function Query(str, search) { - this.str = str; - this.type('and'); - this.search = search; -} - -/** - * Set `type` to "union" or "intersect", aliased as - * "or" and "and". - * - * @param {String} type - * @return {Query} for chaining - * @api public - */ - -Query.prototype.type = function(type){ - this._type = types[type]; - return this; -}; - -/** - * Limit search to the specified range of elements. - * - * @param {String} start - * @param {String} stop - * @return {Query} for chaining - * @api public - */ -Query.prototype.between = function(start, stop){ - this._start = start; - this._stop = stop; - return this; -}; - -/** - * Perform the query and callback `fn(err, ids)`. - * - * @param {Function} fn - * @return {Query} for chaining - * @api public - */ - -Query.prototype.end = function(fn){ - var key = this.search.key; - var db = this.search.client; - var query = this.str; - var nlpObj = this.search.nlpProcess(query, key); - var keys = nlpObj.metaphoneKeys; - var type = this._type; - var start = this._start || 0; - var stop = this._stop || -1; - - if (!keys.length) return fn(null, []); - - var tkey = key + 'tmpkey'; - db.multi([ - [type, tkey, keys.length].concat(keys), - ['zrevrange', tkey, start, stop], - ['zremrangebyrank', tkey, start, stop], - ]).exec(function(err, ids) { - if (err) { - return fn(err) - } - ids = ids[1]; - fn(err, ids); - }); - - return this; -}; - -/** - * Initialize a new `Search` with the given `key`. - * - * @param {String} key - * @api public - */ - -function Search(key, nlpProcess, writeIndex, removeIndex) { - this.key = key; - this.client = exports.createClient(); - - this.nlpProcess = nlpProcess; - this.writeIndex = writeIndex; - this.removeIndex = removeIndex; -} - -/** - * Index the given `str` mapped to `id`. - * - * @param {String} str - * @param {Number|String} id - * @param {Function} fn - * @api public - */ - -Search.prototype.index = function(str, id, fn){ - var key = this.key; - var db = this.client; - var nlpObj = this.nlpProcess(str); - - this.writeIndex(db, id, key, nlpObj, fn); - - return this; -}; - -/** - * Remove occurrences of `id` from the index. - * - * @param {Number|String} id - * @api public - */ - -Search.prototype.remove = function(id, fn){ - fn = fn || noop; - var key = this.key; - var db = this.client; - - this.removeIndex(db, id, key, fn); - - return this; -}; - -/** - * Perform a search on the given `query` returning - * a `Query` instance. - * - * @param {String} query - * @param {Query} - * @api public - */ - -Search.prototype.query = function(query){ - return new Query(query, this); -}; diff --git a/package-lock.json b/package-lock.json new file mode 100644 index 0000000..e879298 --- /dev/null +++ b/package-lock.json @@ -0,0 +1,1215 @@ +{ + "name": "redredisearch", + "version": "0.0.1", + "lockfileVersion": 1, + "requires": true, + "dependencies": { + "accepts": { + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.3.tgz", + "integrity": "sha1-w8p0NJOGSMPg2cHjKN1otiLChMo=", + "dev": true, + "requires": { + "mime-types": "2.1.15", + "negotiator": "0.6.1" + } + }, + "ajv": { + "version": "4.11.8", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-4.11.8.tgz", + "integrity": "sha1-gv+wKynmYq5TvcIK8VlHcGc5xTY=", + "dev": true, + "requires": { + "co": "4.6.0", + "json-stable-stringify": "1.0.1" + } + }, + "ansi-regex": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz", + "integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8=", + "dev": true + }, + "array-flatten": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz", + "integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI=", + "dev": true + }, + "asn1": { + "version": "0.2.3", + "resolved": "https://registry.npmjs.org/asn1/-/asn1-0.2.3.tgz", + "integrity": "sha1-2sh4dxPJlmhJ/IGAd36+nB3fO4Y=", + "dev": true + }, + "assert-plus": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-0.2.0.tgz", + "integrity": "sha1-104bh+ev/A24qttwIfP+SBAasjQ=", + "dev": true + }, + "async": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/async/-/async-2.5.0.tgz", + "integrity": "sha512-e+lJAJeNWuPCNyxZKOBdaJGyLGHugXVQtrAwtuAe2vhxTYxFTKE73p8JuTmdH0qdQZtDvI4dhJwjZc5zsfIsYw==", + "dev": true, + "requires": { + "lodash": "4.17.4" + } + }, + "asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=", + "dev": true + }, + "aws-sign2": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/aws-sign2/-/aws-sign2-0.6.0.tgz", + "integrity": "sha1-FDQt0428yU0OW4fXY81jYSwOeU8=", + "dev": true + }, + "aws4": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/aws4/-/aws4-1.6.0.tgz", + "integrity": "sha1-g+9cqGCysy5KDe7e6MdxudtXRx4=", + "dev": true + }, + "bcrypt-pbkdf": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.1.tgz", + "integrity": "sha1-Y7xdy2EzG5K8Bf1SiVPDNGKgb40=", + "dev": true, + "optional": true, + "requires": { + "tweetnacl": "0.14.5" + } + }, + "boom": { + "version": "2.10.1", + "resolved": "https://registry.npmjs.org/boom/-/boom-2.10.1.tgz", + "integrity": "sha1-OciRjO/1eZ+D+UkqhI9iWt0Mdm8=", + "dev": true, + "requires": { + "hoek": "2.16.3" + } + }, + "builtin-modules": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/builtin-modules/-/builtin-modules-1.1.1.tgz", + "integrity": "sha1-Jw8HbFpywC9bZaR9+Uxf46J4iS8=", + "dev": true + }, + "camelcase": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-3.0.0.tgz", + "integrity": "sha1-MvxLn82vhF/N9+c7uXysImHwqwo=", + "dev": true + }, + "caseless": { + "version": "0.12.0", + "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", + "integrity": "sha1-G2gcIf+EAzyCZUMJBolCDRhxUdw=", + "dev": true + }, + "cliui": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/cliui/-/cliui-3.2.0.tgz", + "integrity": "sha1-EgYBU3qRbSmUD5NNo7SNWFo5IT0=", + "dev": true, + "requires": { + "string-width": "1.0.2", + "strip-ansi": "3.0.1", + "wrap-ansi": "2.1.0" + } + }, + "co": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/co/-/co-4.6.0.tgz", + "integrity": "sha1-bqa989hTrlTMuOR7+gvz+QMfsYQ=", + "dev": true + }, + "code-point-at": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/code-point-at/-/code-point-at-1.1.0.tgz", + "integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c=", + "dev": true + }, + "combined-stream": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.5.tgz", + "integrity": "sha1-k4NwpXtKUd6ix3wV1cX9+JUWQAk=", + "dev": true, + "requires": { + "delayed-stream": "1.0.0" + } + }, + "content-disposition": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz", + "integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ=", + "dev": true + }, + "content-type": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.2.tgz", + "integrity": "sha1-t9ETrueo3Se9IRM8TcJSnfFyHu0=", + "dev": true + }, + "cookie": { + "version": "0.3.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz", + "integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s=", + "dev": true + }, + "cookie-signature": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz", + "integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw=", + "dev": true + }, + "cryptiles": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/cryptiles/-/cryptiles-2.0.5.tgz", + "integrity": "sha1-O9/s3GCBR8HGcgL6KR59ylnqo7g=", + "dev": true, + "requires": { + "boom": "2.10.1" + } + }, + "dashdash": { + "version": "1.14.1", + "resolved": "https://registry.npmjs.org/dashdash/-/dashdash-1.14.1.tgz", + "integrity": "sha1-hTz6D3y+L+1d4gMmuN1YEDX24vA=", + "dev": true, + "requires": { + "assert-plus": "1.0.0" + }, + "dependencies": { + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=", + "dev": true + } + } + }, + "debug": { + "version": "2.6.7", + "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.7.tgz", + "integrity": "sha1-krrR9tBbu2u6Isyoi80OyJTChh4=", + "dev": true, + "requires": { + "ms": "2.0.0" + } + }, + "decamelize": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/decamelize/-/decamelize-1.2.0.tgz", + "integrity": "sha1-9lNNFRSCabIDUue+4m9QH5oZEpA=", + "dev": true + }, + "delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=", + "dev": true + }, + "depd": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.0.tgz", + "integrity": "sha1-4b2Cxqq2ztlluXuIsX7T5SjKGMM=", + "dev": true + }, + "destroy": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz", + "integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA=", + "dev": true + }, + "double-ended-queue": { + "version": "2.1.0-0", + "resolved": "https://registry.npmjs.org/double-ended-queue/-/double-ended-queue-2.1.0-0.tgz", + "integrity": "sha1-ED01J/0xUo9AGIEwyEHv3XgmTlw=" + }, + "drip": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/drip/-/drip-1.1.0.tgz", + "integrity": "sha1-zO+x5obYb8EVtwyewSb4+HG9/X4=", + "dev": true, + "requires": { + "tea-concat": "0.1.0" + } + }, + "ecc-jsbn": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/ecc-jsbn/-/ecc-jsbn-0.1.1.tgz", + "integrity": "sha1-D8c6ntXw1Tw4GTOYUj735UN3dQU=", + "dev": true, + "optional": true, + "requires": { + "jsbn": "0.1.1" + } + }, + "ee-first": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", + "integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0=", + "dev": true + }, + "electron": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/electron/-/electron-0.4.1.tgz", + "integrity": "sha1-p4oFGniC9OVC1uIH2KGnMHazAUQ=", + "dev": true, + "requires": { + "drip": "1.1.0" + } + }, + "encodeurl": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.1.tgz", + "integrity": "sha1-eePVhlU0aQn+bw9Fpd5oEDspTSA=", + "dev": true + }, + "error-ex": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/error-ex/-/error-ex-1.3.1.tgz", + "integrity": "sha1-+FWobOYa3E6GIcPNoh56dhLDqNw=", + "dev": true, + "requires": { + "is-arrayish": "0.2.1" + } + }, + "escape-html": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", + "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg=", + "dev": true + }, + "etag": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.0.tgz", + "integrity": "sha1-b2Ma7zNtbEY2K1F2QETOIWvjwFE=", + "dev": true + }, + "express": { + "version": "4.15.3", + "resolved": "https://registry.npmjs.org/express/-/express-4.15.3.tgz", + "integrity": "sha1-urZdDwOqgMNYQIly/HAPkWlEtmI=", + "dev": true, + "requires": { + "accepts": "1.3.3", + "array-flatten": "1.1.1", + "content-disposition": "0.5.2", + "content-type": "1.0.2", + "cookie": "0.3.1", + "cookie-signature": "1.0.6", + "debug": "2.6.7", + "depd": "1.1.0", + "encodeurl": "1.0.1", + "escape-html": "1.0.3", + "etag": "1.8.0", + "finalhandler": "1.0.3", + "fresh": "0.5.0", + "merge-descriptors": "1.0.1", + "methods": "1.1.2", + "on-finished": "2.3.0", + "parseurl": "1.3.1", + "path-to-regexp": "0.1.7", + "proxy-addr": "1.1.4", + "qs": "6.4.0", + "range-parser": "1.2.0", + "send": "0.15.3", + "serve-static": "1.12.3", + "setprototypeof": "1.0.3", + "statuses": "1.3.1", + "type-is": "1.6.15", + "utils-merge": "1.0.0", + "vary": "1.1.1" + } + }, + "extend": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.1.tgz", + "integrity": "sha1-p1Xqe8Gt/MWjHOfnYtuq3F5jZEQ=", + "dev": true + }, + "extsprintf": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/extsprintf/-/extsprintf-1.0.2.tgz", + "integrity": "sha1-4QgOBljjALBilJkMxw4VAiNf1VA=", + "dev": true + }, + "finalhandler": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.0.3.tgz", + "integrity": "sha1-70fneVDpmXgOhgIqVg4yF+DQzIk=", + "dev": true, + "requires": { + "debug": "2.6.7", + "encodeurl": "1.0.1", + "escape-html": "1.0.3", + "on-finished": "2.3.0", + "parseurl": "1.3.1", + "statuses": "1.3.1", + "unpipe": "1.0.0" + } + }, + "find-up": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-1.1.2.tgz", + "integrity": "sha1-ay6YIrGizgpgq2TWEOzK1TyyTQ8=", + "dev": true, + "requires": { + "path-exists": "2.1.0", + "pinkie-promise": "2.0.1" + } + }, + "forever-agent": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/forever-agent/-/forever-agent-0.6.1.tgz", + "integrity": "sha1-+8cfDEGt6zf5bFd60e1C2P2sypE=", + "dev": true + }, + "form-data": { + "version": "2.1.4", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-2.1.4.tgz", + "integrity": "sha1-M8GDrPGTJ27KqYFDpp6Uv+4XUNE=", + "dev": true, + "requires": { + "asynckit": "0.4.0", + "combined-stream": "1.0.5", + "mime-types": "2.1.15" + } + }, + "forwarded": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.0.tgz", + "integrity": "sha1-Ge+YdMSuHCl7zweP3mOgm2aoQ2M=", + "dev": true + }, + "fresh": { + "version": "0.5.0", + "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.0.tgz", + "integrity": "sha1-9HTKXmqSRtb9jglTz6m5yAWvp44=", + "dev": true + }, + "get-caller-file": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-1.0.2.tgz", + "integrity": "sha1-9wLmMSfn4jHBYKgMFVSstw1QR+U=", + "dev": true + }, + "getpass": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/getpass/-/getpass-0.1.7.tgz", + "integrity": "sha1-Xv+OPmhNVprkyysSgmBOi6YhSfo=", + "dev": true, + "requires": { + "assert-plus": "1.0.0" + }, + "dependencies": { + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=", + "dev": true + } + } + }, + "graceful-fs": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.1.11.tgz", + "integrity": "sha1-Dovf5NHduIVNZOBOp8AOKgJuVlg=", + "dev": true + }, + "har-schema": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/har-schema/-/har-schema-1.0.5.tgz", + "integrity": "sha1-0mMTX0MwfALGAq/I/pWXDAFRNp4=", + "dev": true + }, + "har-validator": { + "version": "4.2.1", + "resolved": "https://registry.npmjs.org/har-validator/-/har-validator-4.2.1.tgz", + "integrity": "sha1-M0gdDxu/9gDdID11gSpqX7oALio=", + "dev": true, + "requires": { + "ajv": "4.11.8", + "har-schema": "1.0.5" + } + }, + "hawk": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/hawk/-/hawk-3.1.3.tgz", + "integrity": "sha1-B4REvXwWQLD+VA0sm3PVlnjo4cQ=", + "dev": true, + "requires": { + "boom": "2.10.1", + "cryptiles": "2.0.5", + "hoek": "2.16.3", + "sntp": "1.0.9" + } + }, + "hoek": { + "version": "2.16.3", + "resolved": "https://registry.npmjs.org/hoek/-/hoek-2.16.3.tgz", + "integrity": "sha1-ILt0A9POo5jpHcRxCo/xuCdKJe0=", + "dev": true + }, + "hosted-git-info": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/hosted-git-info/-/hosted-git-info-2.5.0.tgz", + "integrity": "sha512-pNgbURSuab90KbTqvRPsseaTxOJCZBD0a7t+haSN33piP9cCM4l0CqdzAif2hUqm716UovKB2ROmiabGAKVXyg==", + "dev": true + }, + "http-errors": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.1.tgz", + "integrity": "sha1-X4uO2YrKVFZWv1cplzh/kEpyIlc=", + "dev": true, + "requires": { + "depd": "1.1.0", + "inherits": "2.0.3", + "setprototypeof": "1.0.3", + "statuses": "1.3.1" + } + }, + "http-signature": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/http-signature/-/http-signature-1.1.1.tgz", + "integrity": "sha1-33LiZwZs0Kxn+3at+OE0qPvPkb8=", + "dev": true, + "requires": { + "assert-plus": "0.2.0", + "jsprim": "1.4.0", + "sshpk": "1.13.1" + } + }, + "inherits": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz", + "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=", + "dev": true + }, + "invert-kv": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/invert-kv/-/invert-kv-1.0.0.tgz", + "integrity": "sha1-EEqOSqym09jNFXqO+L+rLXo//bY=", + "dev": true + }, + "ipaddr.js": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.3.0.tgz", + "integrity": "sha1-HgOlL9rYOou7KyXL9JmLTP/NPew=", + "dev": true + }, + "is-arrayish": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/is-arrayish/-/is-arrayish-0.2.1.tgz", + "integrity": "sha1-d8mYQFJ6qOyxqLppe4BkWnqSap0=", + "dev": true + }, + "is-builtin-module": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-builtin-module/-/is-builtin-module-1.0.0.tgz", + "integrity": "sha1-VAVy0096wxGfj3bDDLwbHgN6/74=", + "dev": true, + "requires": { + "builtin-modules": "1.1.1" + } + }, + "is-fullwidth-code-point": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-1.0.0.tgz", + "integrity": "sha1-754xOG8DGn8NZDr4L95QxFfvAMs=", + "dev": true, + "requires": { + "number-is-nan": "1.0.1" + } + }, + "is-typedarray": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-typedarray/-/is-typedarray-1.0.0.tgz", + "integrity": "sha1-5HnICFjfDBsR3dppQPlgEfzaSpo=", + "dev": true + }, + "is-utf8": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/is-utf8/-/is-utf8-0.2.1.tgz", + "integrity": "sha1-Sw2hRCEE0bM2NA6AeX6GXPOffXI=", + "dev": true + }, + "isstream": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/isstream/-/isstream-0.1.2.tgz", + "integrity": "sha1-R+Y/evVa+m+S4VAOaQ64uFKcCZo=", + "dev": true + }, + "jsbn": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/jsbn/-/jsbn-0.1.1.tgz", + "integrity": "sha1-peZUwuWi3rXyAdls77yoDA7y9RM=", + "dev": true, + "optional": true + }, + "json-schema": { + "version": "0.2.3", + "resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.2.3.tgz", + "integrity": "sha1-tIDIkuWaLwWVTOcnvT8qTogvnhM=", + "dev": true + }, + "json-stable-stringify": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/json-stable-stringify/-/json-stable-stringify-1.0.1.tgz", + "integrity": "sha1-mnWdOcXy/1A/1TAGRu1EX4jE+a8=", + "dev": true, + "requires": { + "jsonify": "0.0.0" + } + }, + "json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha1-Epai1Y/UXxmg9s4B1lcB4sc1tus=", + "dev": true + }, + "jsonify": { + "version": "0.0.0", + "resolved": "https://registry.npmjs.org/jsonify/-/jsonify-0.0.0.tgz", + "integrity": "sha1-LHS27kHZPKUbe1qu6PUDYx0lKnM=", + "dev": true + }, + "jsprim": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/jsprim/-/jsprim-1.4.0.tgz", + "integrity": "sha1-o7h+QCmNjDgFUtjMdiigu5WiKRg=", + "dev": true, + "requires": { + "assert-plus": "1.0.0", + "extsprintf": "1.0.2", + "json-schema": "0.2.3", + "verror": "1.3.6" + }, + "dependencies": { + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=", + "dev": true + } + } + }, + "lcid": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/lcid/-/lcid-1.0.0.tgz", + "integrity": "sha1-MIrMr6C8SDo4Z7S28rlQYlHRuDU=", + "dev": true, + "requires": { + "invert-kv": "1.0.0" + } + }, + "load-json-file": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/load-json-file/-/load-json-file-1.1.0.tgz", + "integrity": "sha1-lWkFcI1YtLq0wiYbBPWfMcmTdMA=", + "dev": true, + "requires": { + "graceful-fs": "4.1.11", + "parse-json": "2.2.0", + "pify": "2.3.0", + "pinkie-promise": "2.0.1", + "strip-bom": "2.0.0" + } + }, + "lodash": { + "version": "4.17.4", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.4.tgz", + "integrity": "sha1-eCA6TRwyiuHYbcpkYONptX9AVa4=", + "dev": true + }, + "matcha": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/matcha/-/matcha-0.6.1.tgz", + "integrity": "sha1-ozo7sppb5bGuuBmGmt8IFl6ubKU=", + "dev": true, + "requires": { + "electron": "0.4.1", + "v8-argv": "0.1.0" + } + }, + "media-typer": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", + "integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g=", + "dev": true + }, + "merge-descriptors": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz", + "integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E=", + "dev": true + }, + "methods": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz", + "integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4=", + "dev": true + }, + "mime": { + "version": "1.3.4", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.3.4.tgz", + "integrity": "sha1-EV+eO2s9rylZmDyzjxSaLUDrXVM=", + "dev": true + }, + "mime-db": { + "version": "1.27.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.27.0.tgz", + "integrity": "sha1-gg9XIpa70g7CXtVeW13oaeVDbrE=", + "dev": true + }, + "mime-types": { + "version": "2.1.15", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.15.tgz", + "integrity": "sha1-pOv1BkCUVpI3uM9wBGd20J/JKu0=", + "dev": true, + "requires": { + "mime-db": "1.27.0" + } + }, + "ms": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", + "integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g=", + "dev": true + }, + "negotiator": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz", + "integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk=", + "dev": true + }, + "normalize-package-data": { + "version": "2.4.0", + "resolved": "https://registry.npmjs.org/normalize-package-data/-/normalize-package-data-2.4.0.tgz", + "integrity": "sha512-9jjUFbTPfEy3R/ad/2oNbKtW9Hgovl5O1FvFWKkKblNXoN/Oou6+9+KKohPK13Yc3/TyunyWhJp6gvRNR/PPAw==", + "dev": true, + "requires": { + "hosted-git-info": "2.5.0", + "is-builtin-module": "1.0.0", + "semver": "5.3.0", + "validate-npm-package-license": "3.0.1" + } + }, + "number-is-nan": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz", + "integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0=", + "dev": true + }, + "oauth-sign": { + "version": "0.8.2", + "resolved": "https://registry.npmjs.org/oauth-sign/-/oauth-sign-0.8.2.tgz", + "integrity": "sha1-Rqarfwrq2N6unsBWV4C31O/rnUM=", + "dev": true + }, + "on-finished": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz", + "integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=", + "dev": true, + "requires": { + "ee-first": "1.1.1" + } + }, + "os-locale": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/os-locale/-/os-locale-1.4.0.tgz", + "integrity": "sha1-IPnxeuKe00XoveWDsT0gCYA8FNk=", + "dev": true, + "requires": { + "lcid": "1.0.0" + } + }, + "parse-json": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/parse-json/-/parse-json-2.2.0.tgz", + "integrity": "sha1-9ID0BDTvgHQfhGkJn43qGPVaTck=", + "dev": true, + "requires": { + "error-ex": "1.3.1" + } + }, + "parseurl": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.1.tgz", + "integrity": "sha1-yKuMkiO6NIiKpkopeyiFO+wY2lY=", + "dev": true + }, + "path-exists": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-2.1.0.tgz", + "integrity": "sha1-D+tsZPD8UY2adU3V77YscCJ2H0s=", + "dev": true, + "requires": { + "pinkie-promise": "2.0.1" + } + }, + "path-to-regexp": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz", + "integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w=", + "dev": true + }, + "path-type": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/path-type/-/path-type-1.1.0.tgz", + "integrity": "sha1-WcRPfuSR2nBNpBXaWkBwuk+P5EE=", + "dev": true, + "requires": { + "graceful-fs": "4.1.11", + "pify": "2.3.0", + "pinkie-promise": "2.0.1" + } + }, + "performance-now": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/performance-now/-/performance-now-0.2.0.tgz", + "integrity": "sha1-M+8wxcd9TqIcWlOGnZG1bY8lVeU=", + "dev": true + }, + "pify": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/pify/-/pify-2.3.0.tgz", + "integrity": "sha1-7RQaasBDqEnqWISY59yosVMw6Qw=", + "dev": true + }, + "pinkie": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/pinkie/-/pinkie-2.0.4.tgz", + "integrity": "sha1-clVrgM+g1IqXToDnckjoDtT3+HA=", + "dev": true + }, + "pinkie-promise": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/pinkie-promise/-/pinkie-promise-2.0.1.tgz", + "integrity": "sha1-ITXW36ejWMBprJsXh3YogihFD/o=", + "dev": true, + "requires": { + "pinkie": "2.0.4" + } + }, + "proxy-addr": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-1.1.4.tgz", + "integrity": "sha1-J+VF9pYKRKYn2bREZ+NcG2tM4vM=", + "dev": true, + "requires": { + "forwarded": "0.1.0", + "ipaddr.js": "1.3.0" + } + }, + "punycode": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/punycode/-/punycode-1.4.1.tgz", + "integrity": "sha1-wNWmOycYgArY4esPpSachN1BhF4=", + "dev": true + }, + "qs": { + "version": "6.4.0", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.4.0.tgz", + "integrity": "sha1-E+JtKK1rD/qpExLNO/cI7TUecjM=", + "dev": true + }, + "range-parser": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz", + "integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4=", + "dev": true + }, + "read-pkg": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/read-pkg/-/read-pkg-1.1.0.tgz", + "integrity": "sha1-9f+qXs0pyzHAR0vKfXVra7KePyg=", + "dev": true, + "requires": { + "load-json-file": "1.1.0", + "normalize-package-data": "2.4.0", + "path-type": "1.1.0" + } + }, + "read-pkg-up": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/read-pkg-up/-/read-pkg-up-1.0.1.tgz", + "integrity": "sha1-nWPBMnbAZZGNV/ACpX9AobZD+wI=", + "dev": true, + "requires": { + "find-up": "1.1.2", + "read-pkg": "1.1.0" + } + }, + "redis": { + "version": "2.7.1", + "resolved": "https://registry.npmjs.org/redis/-/redis-2.7.1.tgz", + "integrity": "sha1-fVb3h1uYsgQQtxU58dh47Vjr9Go=", + "requires": { + "double-ended-queue": "2.1.0-0", + "redis-commands": "1.3.1", + "redis-parser": "2.6.0" + } + }, + "redis-commands": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/redis-commands/-/redis-commands-1.3.1.tgz", + "integrity": "sha1-gdgm9F+pyLIBH0zXoP5ZfSQdRCs=" + }, + "redis-parser": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-2.6.0.tgz", + "integrity": "sha1-Uu0J2srBCPGmMcB+m2mUHnoZUEs=" + }, + "request": { + "version": "2.81.0", + "resolved": "https://registry.npmjs.org/request/-/request-2.81.0.tgz", + "integrity": "sha1-xpKJRqDgbF+Nb4qTM0af/aRimKA=", + "dev": true, + "requires": { + "aws-sign2": "0.6.0", + "aws4": "1.6.0", + "caseless": "0.12.0", + "combined-stream": "1.0.5", + "extend": "3.0.1", + "forever-agent": "0.6.1", + "form-data": "2.1.4", + "har-validator": "4.2.1", + "hawk": "3.1.3", + "http-signature": "1.1.1", + "is-typedarray": "1.0.0", + "isstream": "0.1.2", + "json-stringify-safe": "5.0.1", + "mime-types": "2.1.15", + "oauth-sign": "0.8.2", + "performance-now": "0.2.0", + "qs": "6.4.0", + "safe-buffer": "5.1.1", + "stringstream": "0.0.5", + "tough-cookie": "2.3.2", + "tunnel-agent": "0.6.0", + "uuid": "3.1.0" + } + }, + "require-directory": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", + "integrity": "sha1-jGStX9MNqxyXbiNE/+f3kqam30I=", + "dev": true + }, + "require-main-filename": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/require-main-filename/-/require-main-filename-1.0.1.tgz", + "integrity": "sha1-l/cXtp1IeE9fUmpsWqj/3aBVpNE=", + "dev": true + }, + "safe-buffer": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz", + "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg==", + "dev": true + }, + "semver": { + "version": "5.3.0", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.3.0.tgz", + "integrity": "sha1-myzl094C0XxgEq0yaqa00M9U+U8=", + "dev": true + }, + "send": { + "version": "0.15.3", + "resolved": "https://registry.npmjs.org/send/-/send-0.15.3.tgz", + "integrity": "sha1-UBP5+ZAj31DRvZiSwZ4979HVMwk=", + "dev": true, + "requires": { + "debug": "2.6.7", + "depd": "1.1.0", + "destroy": "1.0.4", + "encodeurl": "1.0.1", + "escape-html": "1.0.3", + "etag": "1.8.0", + "fresh": "0.5.0", + "http-errors": "1.6.1", + "mime": "1.3.4", + "ms": "2.0.0", + "on-finished": "2.3.0", + "range-parser": "1.2.0", + "statuses": "1.3.1" + } + }, + "serve-static": { + "version": "1.12.3", + "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.12.3.tgz", + "integrity": "sha1-n0uhni8wMMVH+K+ZEHg47DjVseI=", + "dev": true, + "requires": { + "encodeurl": "1.0.1", + "escape-html": "1.0.3", + "parseurl": "1.3.1", + "send": "0.15.3" + } + }, + "set-blocking": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/set-blocking/-/set-blocking-2.0.0.tgz", + "integrity": "sha1-BF+XgtARrppoA93TgrJDkrPYkPc=", + "dev": true + }, + "setprototypeof": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz", + "integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ=", + "dev": true + }, + "should": { + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/should/-/should-3.3.2.tgz", + "integrity": "sha1-yIPdQJtTu98bVewNj8OGXysofmQ=", + "dev": true + }, + "sntp": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/sntp/-/sntp-1.0.9.tgz", + "integrity": "sha1-ZUEYTMkK7qbG57NeJlkIJEPGYZg=", + "dev": true, + "requires": { + "hoek": "2.16.3" + } + }, + "spdx-correct": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/spdx-correct/-/spdx-correct-1.0.2.tgz", + "integrity": "sha1-SzBz2TP/UfORLwOsVRlJikFQ20A=", + "dev": true, + "requires": { + "spdx-license-ids": "1.2.2" + } + }, + "spdx-expression-parse": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/spdx-expression-parse/-/spdx-expression-parse-1.0.4.tgz", + "integrity": "sha1-m98vIOH0DtRH++JzJmGR/O1RYmw=", + "dev": true + }, + "spdx-license-ids": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/spdx-license-ids/-/spdx-license-ids-1.2.2.tgz", + "integrity": "sha1-yd96NCRZSt5r0RkA1ZZpbcBrrFc=", + "dev": true + }, + "sshpk": { + "version": "1.13.1", + "resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.13.1.tgz", + "integrity": "sha1-US322mKHFEMW3EwY/hzx2UBzm+M=", + "dev": true, + "requires": { + "asn1": "0.2.3", + "assert-plus": "1.0.0", + "bcrypt-pbkdf": "1.0.1", + "dashdash": "1.14.1", + "ecc-jsbn": "0.1.1", + "getpass": "0.1.7", + "jsbn": "0.1.1", + "tweetnacl": "0.14.5" + }, + "dependencies": { + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=", + "dev": true + } + } + }, + "statuses": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.3.1.tgz", + "integrity": "sha1-+vUbnrdKrvOzrPStX2Gr8ky3uT4=", + "dev": true + }, + "string-width": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/string-width/-/string-width-1.0.2.tgz", + "integrity": "sha1-EYvfW4zcUaKn5w0hHgfisLmxB9M=", + "dev": true, + "requires": { + "code-point-at": "1.1.0", + "is-fullwidth-code-point": "1.0.0", + "strip-ansi": "3.0.1" + } + }, + "stringstream": { + "version": "0.0.5", + "resolved": "https://registry.npmjs.org/stringstream/-/stringstream-0.0.5.tgz", + "integrity": "sha1-TkhM1N5aC7vuGORjB3EKioFiGHg=", + "dev": true + }, + "strip-ansi": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz", + "integrity": "sha1-ajhfuIU9lS1f8F0Oiq+UJ43GPc8=", + "dev": true, + "requires": { + "ansi-regex": "2.1.1" + } + }, + "strip-bom": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/strip-bom/-/strip-bom-2.0.0.tgz", + "integrity": "sha1-YhmoVhZSBJHzV4i9vxRHqZx+aw4=", + "dev": true, + "requires": { + "is-utf8": "0.2.1" + } + }, + "tea-concat": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/tea-concat/-/tea-concat-0.1.0.tgz", + "integrity": "sha1-6i6QdAD914pjNM4CD6PGlQLZnoQ=", + "dev": true + }, + "tough-cookie": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-2.3.2.tgz", + "integrity": "sha1-8IH3bkyFcg5sN6X6ztc3FQ2EByo=", + "dev": true, + "requires": { + "punycode": "1.4.1" + } + }, + "tunnel-agent": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz", + "integrity": "sha1-J6XeoGs2sEoKmWZ3SykIaPD8QP0=", + "dev": true, + "requires": { + "safe-buffer": "5.1.1" + } + }, + "tweetnacl": { + "version": "0.14.5", + "resolved": "https://registry.npmjs.org/tweetnacl/-/tweetnacl-0.14.5.tgz", + "integrity": "sha1-WuaBd/GS1EViadEIr6k/+HQ/T2Q=", + "dev": true, + "optional": true + }, + "type-is": { + "version": "1.6.15", + "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.15.tgz", + "integrity": "sha1-yrEPtJCeRByChC6v4a1kbIGARBA=", + "dev": true, + "requires": { + "media-typer": "0.3.0", + "mime-types": "2.1.15" + } + }, + "unpipe": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz", + "integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw=", + "dev": true + }, + "utils-merge": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.0.tgz", + "integrity": "sha1-ApT7kiu5N1FTVBxPcJYjHyh8ivg=", + "dev": true + }, + "uuid": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.1.0.tgz", + "integrity": "sha512-DIWtzUkw04M4k3bf1IcpS2tngXEL26YUD2M0tMDUpnUrz2hgzUBlD55a4FjdLGPvfHxS6uluGWvaVEqgBcVa+g==", + "dev": true + }, + "v8-argv": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/v8-argv/-/v8-argv-0.1.0.tgz", + "integrity": "sha1-rfd3pS29w9qciclGXlntXzy22ak=", + "dev": true + }, + "validate-npm-package-license": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/validate-npm-package-license/-/validate-npm-package-license-3.0.1.tgz", + "integrity": "sha1-KAS6vnEq0zeUWaz74kdGqywwP7w=", + "dev": true, + "requires": { + "spdx-correct": "1.0.2", + "spdx-expression-parse": "1.0.4" + } + }, + "vary": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.1.tgz", + "integrity": "sha1-Z1Neu2lMHVIldFeYRmUyP1h+jTc=", + "dev": true + }, + "verror": { + "version": "1.3.6", + "resolved": "https://registry.npmjs.org/verror/-/verror-1.3.6.tgz", + "integrity": "sha1-z/XfEpRtKX0rqu+qJoniW+AcAFw=", + "dev": true, + "requires": { + "extsprintf": "1.0.2" + } + }, + "which-module": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/which-module/-/which-module-1.0.0.tgz", + "integrity": "sha1-u6Y8qGGUiZT/MHc2CJ47lgJsKk8=", + "dev": true + }, + "wrap-ansi": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-2.1.0.tgz", + "integrity": "sha1-2Pw9KE3QV5T+hJc8rs3Rz4JP3YU=", + "dev": true, + "requires": { + "string-width": "1.0.2", + "strip-ansi": "3.0.1" + } + }, + "y18n": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/y18n/-/y18n-3.2.1.tgz", + "integrity": "sha1-bRX7qITAhnnA136I53WegR4H+kE=", + "dev": true + }, + "yargs": { + "version": "7.1.0", + "resolved": "https://registry.npmjs.org/yargs/-/yargs-7.1.0.tgz", + "integrity": "sha1-a6MY6xaWFyf10oT46gA+jWFU0Mg=", + "dev": true, + "requires": { + "camelcase": "3.0.0", + "cliui": "3.2.0", + "decamelize": "1.2.0", + "get-caller-file": "1.0.2", + "os-locale": "1.4.0", + "read-pkg-up": "1.0.1", + "require-directory": "2.1.1", + "require-main-filename": "1.0.1", + "set-blocking": "2.0.0", + "string-width": "1.0.2", + "which-module": "1.0.0", + "y18n": "3.2.1", + "yargs-parser": "5.0.0" + } + }, + "yargs-parser": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-5.0.0.tgz", + "integrity": "sha1-J17PDX/+Bcd+ZOfIbkzZS/DhIoo=", + "dev": true, + "requires": { + "camelcase": "3.0.0" + } + } + } +} diff --git a/package.json b/package.json index f5c7a3a..097a259 100644 --- a/package.json +++ b/package.json @@ -1,17 +1,14 @@ { - "name": "reds", - "version": "1.0.0", - "description": "Redis search for node.js", + "name": "redredisearch", + "version": "0.0.1", + "description": "Redis search for node.js powered by the RediSearch module", "keywords": [ "redis", "search", - "metaphone", - "phonetics", - "natural" + "redisearch" ], - "author": "TJ Holowaychuk ", + "author": "Kyle Davis", "dependencies": { - "natural": "0.5.0", "redis": "^2.7.1" }, "devDependencies": { @@ -19,11 +16,12 @@ "async": "^2.3.0", "matcha": "^0.6.0", "should": "^3.3.2", - "superagent": "^0.21.0" + "request": "^2.81.0", + "express": "^4.15.3" }, "main": "index", "repository": { "type": "git", - "url": "https://github.com/visionmedia/reds.git" + "url": "https://github.com/stockholmux/redredisearch" } } diff --git a/test/index.js b/test/index.js deleted file mode 100644 index 93d6b34..0000000 --- a/test/index.js +++ /dev/null @@ -1,367 +0,0 @@ - -/** - * Module dependencies. - */ - -var async = require('async') - , reds = require('../') - , should = require('should') - , redis = require('redis') - - , argv = require('yargs') - .demand('connection') - .argv - , connectionObj = require(argv.connection); - -reds.version.should.match(/^\d+\.\d+\.\d+$/); -reds.setClient(redis.createClient(connectionObj)); //two clients - -function nonModularTests(finish) { - var db = redis.createClient(connectionObj) - , search - , start = new Date() - , test - , done; - - console.log('Starting non-modular (original) test'); - - done = function() { - console.log(' completed in %dms', new Date() - start); - db.quit(); - finish(); - }; - - test = function() { - var pending = 0; - - ++pending; - search - .query('stuff compute') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['6']); - --pending || done(); - }); - - ++pending; - search - .query('Tobi') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(3); - ids.should.include('0'); - ids.should.include('3'); - ids.should.include('5'); - --pending || done(); - }); - - ++pending; - search - .query('tobi') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(3); - ids.should.include('0'); - ids.should.include('3'); - ids.should.include('5'); - --pending || done(); - }); - - ++pending; - search - .query('bitchy') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['4']); - --pending || done(); - }); - - ++pending; - search - .query('bitchy jane') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['4']); - --pending || done(); - }); - - ++pending; - search - .query('loki and jane') - .type('or') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(2); - ids.should.include('2'); - ids.should.include('4'); - --pending || done(); - }); - - ++pending; - search - .query('loki and jane') - .type('or') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(2); - ids.should.include('2'); - ids.should.include('4'); - --pending || done(); - }); - - ++pending; - search - .query('loki and jane') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql([]); - --pending || done(); - }); - - ++pending; - search - .query('jane ferret') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['4']); - --pending || done(); - }); - - ++pending; - search - .query('is a') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.eql([]); - --pending || done(); - }); - - ++pending; - search - .query('simple') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(2); - ids.should.include('7'); - ids.should.include('9'); - ids[0].should.eql('7'); - ids[1].should.eql('9'); - --pending || done(); - }); - - ++pending; - search - .query('dog ideas') - .type('or') - .end(function(err, ids){ - if (err) { throw err; } - ids.should.have.length(3); - ids.should.include('7'); - ids.should.include('8'); - ids.should.include('9'); - ids[0].should.eql('9'); - --pending || done(); - }); - - ++pending; - //refactor this with async soon - search - .index('keyboard cat', 6, function(err){ - if (err) { throw err; } - search.query('keyboard').end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['6']); - search.query('cat').end(function(err, ids){ - if (err) { throw err; } - ids.should.eql(['6']); - search.remove(6, function(err){ - if (err) { throw err; } - search.query('keyboard').end(function(err, ids){ - if (err) { throw err; } - ids.should.be.empty; - search.query('cat').end(function(err, ids){ - if (err) { throw err; } - ids.should.be.empty; - --pending || done(); - }); - }); - }); - }); - }); - }); - } - - - search = reds.createSearch('reds'); - - reds - .words('foo bar baz ') - .should.eql(['foo', 'bar', 'baz']); - - reds - .words(' Punctuation and whitespace; should be, handled.') - .should.eql(['Punctuation', 'and', 'whitespace', 'should', 'be', 'handled']); - - reds - .stripStopWords(['this', 'is', 'just', 'a', 'test']) - .should.eql(['just', 'test']); - - reds - .countWords(['foo', 'bar', 'baz', 'foo', 'jaz', 'foo', 'baz']) - .should.eql({ - foo: 3 - , bar: 1 - , baz: 2 - , jaz: 1 - }); - - reds - .metaphoneMap(['foo', 'bar', 'baz']) - .should.eql({ - foo: 'F' - , bar: 'BR' - , baz: 'BS' - }); - - reds - .metaphoneArray(['foo', 'bar', 'baz']) - .should.eql(['F', 'BR', 'BS']); - - reds - .metaphoneKeys('reds', ['foo', 'bar', 'baz']) - .should.eql(['reds:word:F', 'reds:word:BR', 'reds:word:BS']); - - reds - .metaphoneKeys('foobar', ['foo', 'bar', 'baz']) - .should.eql(['foobar:word:F', 'foobar:word:BR', 'foobar:word:BS']); - - db.flushdb(function(){ - search - .index('Tobi wants 4 dollars', 0) - .index('Loki is a ferret', 2) - .index('Tobi is also a ferret', 3) - .index('Jane is a bitchy ferret', 4) - .index('Tobi is employed by LearnBoost', 5) - .index('computing stuff', 6) - .index('simple words do not mean simple ideas', 7) - .index('The dog spoke the words, much to our unbelief.', 8) - .index('puppy dog eagle puppy frog puppy dog simple', 9, test); - }); -} - -function modularTest(next) { - var - start = new Date(), - writeIndexTestProp = 'rand'+Math.random(), - deleteIndexTestProp = 'delete-'+writeIndexTestProp, - testString = 'trent toronto', - search, - /* this is an example of doing a non-phoentic implementation */ - nonPhoneticKeys = function(key, words){ - return words.map(function(c){ - return key + ':word:' + c.toUpperCase(); - }); - }, - nonPhoneticMap = function(words){ - var - obj = {}, - len, - i; - if (!words) { return obj; } else { - len = words.length; - for (i = 0; i < len; i += 1) { - obj[words[i]] = words[i].toUpperCase(); - } - return obj; - } - }, - //normally, the natural language processing routine in reds will both stem and convert the stemmed words to metaphones - //here we've replaced it with the simplifed `customProcessor` which only stems, but no metaphone processing - customProcessor = function(str, key) { - var words = reds.stem(reds.stripStopWords(reds.words(str))); - var counts = reds.countWords(words); - var map = nonPhoneticMap(words); - var keys = Object.keys(map); - var metaphoneKeys = !key ? null : nonPhoneticKeys(key, words); - - return { - words : words, - counts : counts, - map : map, - keys : keys, - metaphoneKeys - : metaphoneKeys - }; - }; - - - - console.log('Starting modular test'); - reds.client.quit(); - - //this test is not using the redis connection by effectively dumbing the writeIndex and removeIndex, - //we then know that it's using the modular functions. - reds.setClient('this is not valid'); - - search = reds.createSearch('reds',{ - nlpProcess : customProcessor, - writeIndex : function(db, id, key, nlpObj) { - var cmds = []; - nlpObj.keys.forEach(function(word){ - cmds.push(['zadd', key + ':word:' + nlpObj.map[word], nlpObj.counts[word], id]); - cmds.push(['zadd', key + ':object:' + id, nlpObj.counts[word], nlpObj.map[word]]); - }); - //we actually aren't writing anything to redis here - this will only work work if the writeIndex is being invoked from here not from the default one. - this[writeIndexTestProp] = cmds; - }, - removeIndex : function() { - this[deleteIndexTestProp] = true; - } - }); - - search.index(testString,'x'); - - should.exist(search[writeIndexTestProp]) - - //with the normal nlpProcessor, 'trent' and 'toronto' will turn into the same metaphones, test ot make sure that isn't happening. - search[writeIndexTestProp] - .should - .eql([ [ 'zadd', 'reds:word:TRENT', 1, 'x' ], - [ 'zadd', 'reds:object:x', 1, 'TRENT' ], - [ 'zadd', 'reds:word:TORONTO', 1, 'x' ], - [ 'zadd', 'reds:object:x', 1, 'TORONTO' ] ]); - - search.remove('trent toronto','x'); - - should.exist(search[deleteIndexTestProp]); - - - reds.setClient(redis.createClient(connectionObj)); - search = reds.createSearch('reds-custom',{ - nlpProcess : customProcessor - }); - - search - .index('trent','x') - .index('toronto','y', function() { - search - .query('trent') - .end(function(err,results) { - if (err) { throw err; } - results.should.eql(['x']); //if it was using the standard nlpParser then it would be ['x','y'] - console.log(' completed in %dms', new Date() - start); - - next(); - }); - }); -} - -function done() { - console.log('All done.'); - reds.client.quit(); -} - -async.series([ - nonModularTests, - modularTest -], done);