19:1cf3b82959f3
|
2009-09-02 |
Paul Boddie |
changeset
files
shortlog
graph
|
Attempted to introduce position dictionaries with extra term record details
providing document frequency information.
Attempted to introduce file descriptor duplication in order to support
concurrent iterators. |
|
|
iixr.py test.py
|
|
18:1e7ca36202ef
|
2009-08-31 |
Paul Boddie |
changeset
files
shortlog
graph
|
Introduced a positions iterator to permit incremental reading of position
information. |
|
|
iixr.py test.py
|
|
17:e6f4e8e226a0
|
2009-08-31 |
Paul Boddie |
changeset
files
shortlog
graph
|
Fixed copying error. |
|
|
iixr.py
|
|
16:fb0ad812b19f
|
2009-08-31 |
Paul Boddie |
changeset
files
shortlog
graph
|
Ensure document position ordering at the lowest level. |
|
|
iixr.py
|
|
15:ae07d4c3c9af
|
2009-08-30 |
Paul Boddie |
changeset
files
shortlog
graph
|
Fixed counters used for determining when to flush written data. |
|
|
iixr.py
|
|
14:51cf2f142879
|
2009-08-30 |
Paul Boddie |
changeset
files
shortlog
graph
|
Fixed term discovery in the term dictionary index within the _find_term method.
Fixed various typing errors in the merging functionality.
Introduced merging into the process of obtaining a reader from top-level Index
instances: only merged dictionaries should be accessed.
Introduced the renaming of single partitions for reading and the deletion of
partitions after merging.
Renamed "index-N" to "terms_index-N" for term dictionary index files.
Moved dictionary reader acquisition to utility functions, and changed merging
methods to use such readers directly.
Introduced merging into the test program. |
|
|
iixr.py test.py
|
|
13:eafc8a8a84ff
|
2009-08-30 |
Paul Boddie |
changeset
files
shortlog
graph
|
Made the rewind method for FileReader use the reset method.
Added field identifiers, permitting sensible merging behaviour for field
dictionaries.
Changed the field writing method in the field dictionary writer from add_fields
to add_field, employing an explicit field identifier.
Added sequential field reading support in the field dictionary reader.
Added more support for merging dictionaries.
Created utility functions for opening term and field dictionary writers.
Changed filename construction to permit strings as partition identifiers. |
|
|
iixr.py test.py
|
|
12:0ba1bf2fa563
|
2009-08-30 |
Paul Boddie |
changeset
files
shortlog
graph
|
Introduced index "partitions", sequential access to term dictionaries, and some
support for merging partitions. |
|
|
iixr.py test.py
|
|
11:3d86f5cb01c1
|
2009-08-29 |
Paul Boddie |
changeset
files
shortlog
graph
|
Added term frequency information to the term dictionary. |
|
|
iixr.py test.py
|
|
10:b81c00a48c49
|
2009-08-29 |
Paul Boddie |
changeset
files
shortlog
graph
|
Introduced conditional compression for fields using bzip2 and zlib compression.
Added an IndexReader class to encapsulate all reading operations (using term and
field dictionaries).
Added field-related file operations to the IndexWriter class.
Added field-related file initialisation to the Index class.
Changed the field index format to use offset deltas. |
|
|
iixr.py test.py
|
|