|
Zip and Skip Tries
|
A class to store strings compactly into integers, packing multiple characters into a single word. More...
Classes | |
| class | Iterator |
| An iterator class for traversing the characters within a BitString. More... | |
| struct | ResultLCP |
| Structure to hold the result of a comparison, including the ordering and the length of the common prefix. More... | |
Public Member Functions | |
| BitString (const CHAR_T *data, size_t size) | |
| Constructs a BitString object from a raw character array. | |
| template<typename Container > | |
| BitString (const Container &data) | |
| Constructs a BitString object from a container of characters. | |
| BitString (const std::string &data) | |
Constructs a BitString object from a std::string. | |
| BitString ()=default | |
| Default constructor. | |
| void | push_back (CHAR_T c) noexcept |
| Appends a single character to the end of the BitString. | |
| CHAR_T | operator[] (size_t index) const |
| Accesses the character at a specific index. | |
| size_t | size () const noexcept |
| Returns the number of characters stored in the BitString. | |
| size_t | word_count () const noexcept |
Returns the number of uintmax_t words used for storage. | |
| void | clear () noexcept |
| Clears the contents of the BitString, making it empty. | |
| ResultLCP | seq_k_compare (const BitString &other, size_t lcp, size_t max_compare) const noexcept |
| Performs sequential comparison with another BitString, starting from a given LCP offset, up to a maximum length. | |
| ResultLCP | seq_k_compare (const BitString &other, size_t lcp) const noexcept |
| Performs sequential comparison with another BitString, starting from a given LCP offset. | |
| ResultLCP | par_k_compare (const BitString &other, size_t lcp, size_t max_compare, uintmax_t *d_a, uintmax_t *d_largeblock, size_t &max_copied) const noexcept |
| Performs parallel comparison (potentially using CUDA) with another BitString, starting from a given LCP offset, up to a maximum length. | |
| ResultLCP | par_k_compare (const BitString &other, size_t lcp, uintmax_t *d_a, uintmax_t *d_largeblock, size_t &max_copied) const noexcept |
| Performs parallel comparison (potentially using CUDA) with another BitString, starting from a given LCP offset. | |
| const uintmax_t * | data () const noexcept |
| Provides direct access to the underlying packed data array. | |
| void | print_bytes (std::ostream &os) const |
| Prints the raw byte representation of the BitString to an output stream. | |
| void | to_file (const std::string &filename) const |
| Writes the BitString contents to a file in binary format. | |
| void | to_file (std::ofstream &file) const |
| Writes the BitString contents to an already open output file stream. | |
| std::string | to_string () const noexcept |
Converts the BitString back to a standard std::string. | |
| Iterator | begin () const noexcept |
| Returns an iterator pointing to the beginning of the BitString. | |
| Iterator | end () const noexcept |
| Returns an iterator pointing past the end of the BitString. | |
Static Public Member Functions | |
| static BitString | from_file (const std::string &filename) |
| Reads a BitString from a file in binary format. | |
| static BitString | from_file (std::ifstream &file) |
| Reads a BitString from an already open input file stream. | |
Static Public Attributes | |
| static constexpr unsigned | WORD_SIZE_BITS = sizeof(uintmax_t) * 8 |
Size of the underlying storage word (uintmax_t) in bits. | |
| static constexpr unsigned | ALPHA = WORD_SIZE_BITS / CHAR_SIZE_BITS |
| Number of characters that can be packed into a single storage word. | |
| static constexpr size_t | MIN_PAR_COMPARE_CHAR_SIZE = MIN_PAR_COMPARE_WORD_SIZE * ALPHA |
Minimum number of characters required to trigger parallel comparison. Derived from MIN_PAR_COMPARE_WORD_SIZE. | |
Static Private Member Functions | |
| static constexpr std::array< unsigned, ALPHA > | GET_SHIFTS () |
| Generates the bit shift amounts needed for character extraction/insertion at compile time. | |
| static constexpr std::array< uintmax_t, ALPHA > | GET_MASKS () |
| Generates the bit masks needed for character extraction at compile time. | |
| static constexpr CHAR_T | GET_CHAR (uintmax_t word, unsigned bit_index) |
| Extracts a single character from a given word at a specific bit index (sub-word position). | |
Private Attributes | |
| std::vector< uintmax_t > | m_data |
| Internal storage vector holding the packed character data. | |
| size_t | m_size = 0 |
| The number of characters stored in the BitString. | |
Static Private Attributes | |
| static constexpr std::array< unsigned, ALPHA > | m_shifts = GET_SHIFTS() |
| Compile-time generated array of bit shift amounts. | |
| static constexpr std::array< uintmax_t, ALPHA > | m_masks = GET_MASKS() |
| Compile-time generated array of bit masks. | |
Friends | |
| template<typename CT , unsigned CSB> | |
| std::ostream & | operator<< (std::ostream &os, const BitString< CT, CSB > &bs) |
A class to store strings compactly into integers, packing multiple characters into a single word.
This class allows for space-efficient storage of character strings by packing them into an array of uintmax_t integers. It focuses on optimizing storage and comparison operations. Comparisons leverage bitwise operations and can identify the longest common prefix (LCP) efficiently. Parallel comparison using CUDA is supported for large strings.
| CHAR_T | The character type to be stored (e.g., char, uint8_t, wchar_t). |
| CHAR_SIZE_BITS | The number of bits representing relevant data within CHAR_T. Defaults to the full size of CHAR_T in bits. This allows handling types where only a subset of bits are meaningful (e.g., 7-bit ASCII in an 8-bit char, or 2-bit nucleotides for DNA strings). |
Definition at line 57 of file BitString.cuh.
|
explicit |
Constructs a BitString object from a raw character array.
Packs the input characters into the internal uintmax_t vector.
| data | Pointer to the beginning of the character array. Must not be null if size > 0. |
| size | The number of characters in the array. |
Definition at line 491 of file BitString.cuh.
|
explicit |
Constructs a BitString object from a container of characters.
Packs characters from the container into the internal vector.
The container must support size() and random access via operator[]. Examples include std::vector<CHAR_T>, std::basic_string<CHAR_T>, std::array<CHAR_T, N>.
| Container | Type of the container holding the character data. |
| data | The container instance. |
Definition at line 508 of file BitString.cuh.
|
inlineexplicit |
Constructs a BitString object from a std::string.
CHAR_T is compatible with char. | data | The std::string used to initialize the BitString object. |
Definition at line 93 of file BitString.cuh.
|
default |
Default constructor.
from_file static method.
|
inlinenoexcept |
Returns an iterator pointing to the beginning of the BitString.
Definition at line 437 of file BitString.cuh.
|
inlinenoexcept |
Clears the contents of the BitString, making it empty.
Definition at line 141 of file BitString.cuh.
|
inlinenoexcept |
Provides direct access to the underlying packed data array.
uintmax_t data array. Returns nullptr if the BitString is empty. Definition at line 246 of file BitString.cuh.
|
inlinenoexcept |
Returns an iterator pointing past the end of the BitString.
Definition at line 446 of file BitString.cuh.
|
static |
Reads a BitString from a file in binary format.
Opens the file, calls the stream version of from_file, and closes the file.
Deserializes the size and packed data from the specified file.
| filename | The path to the input file. |
| std::runtime_error | If the file cannot be opened for reading. |
Definition at line 763 of file BitString.cuh.
|
static |
Reads a BitString from an already open input file stream.
Reads the size, calculates words needed, resizes, then reads raw data. Does not perform error checking on reads.
Deserializes the size and packed data from the stream.
| file | The std::ifstream object, opened in binary mode. |
Definition at line 739 of file BitString.cuh.
|
staticconstexprprivate |
Extracts a single character from a given word at a specific bit index (sub-word position).
Applies the appropriate mask and shift to isolate and retrieve the character.
| CHAR_T | Character type. |
| CHAR_SIZE_BITS | Bits per character. |
| word | The uintmax_t word containing the packed characters. |
| bit_index | The index within the word (0 to ALPHA-1) corresponding to the desired character. |
Definition at line 483 of file BitString.cuh.
|
staticconstexprprivate |
Generates the bit masks needed for character extraction at compile time.
Creates a mask for each character position within a word.
| CHAR_T | Character type. |
| CHAR_SIZE_BITS | Bits per character. |
i is a mask that isolates the bits corresponding to the character at sub-word index i. Definition at line 466 of file BitString.cuh.
|
staticconstexprprivate |
Generates the bit shift amounts needed for character extraction/insertion at compile time.
Calculates the left shift amount needed for each character position within a word.
| CHAR_T | Character type. |
| CHAR_SIZE_BITS | Bits per character. |
i contains the left-shift amount required to position the character at sub-word index i correctly within a uintmax_t. Definition at line 455 of file BitString.cuh.
| CHAR_T BitString< CHAR_T, CHAR_SIZE_BITS >::operator[] | ( | size_t | index | ) | const |
Accesses the character at a specific index.
Calculates the word and sub-index, then uses GET_CHAR for extraction. Behavior undefined if index is out of bounds.
Behavior is undefined for out-of-bounds access.
| index | The zero-based index of the character to access. Must be less than size(). |
Definition at line 540 of file BitString.cuh.
|
noexcept |
Performs parallel comparison (potentially using CUDA) with another BitString, starting from a given LCP offset, up to a maximum length.
Checks size, copies data to GPU if necessary, calls parallel mismatch kernel, then refines LCP. Falls back to sequential if too small.
Compares this BitString with other, assuming the first lcp characters are equal. If the number of words to compare exceeds MIN_PAR_COMPARE_WORD_SIZE, this function attempts to use a parallel CUDA kernel (par_find_mismatch) for faster mismatch detection. Otherwise, it falls back to seq_k_compare. The comparison stops strictly at lcp + max_compare characters or the end of the shorter string, whichever comes first.
| other | The BitString to compare against. | |
| lcp | The starting offset (longest common prefix length) for the comparison, in characters. | |
| max_compare | The maximum number of additional characters to compare beyond the initial lcp. The comparison will not proceed beyond lcp + max_compare total characters from the start. | |
| d_a | Pointer to device (GPU) memory allocated for this BitString's data. Data might be copied here if needed. | |
| d_largeblock | Pointer to auxiliary device (GPU) memory required by the parallel kernel. | |
| [in,out] | max_copied | Tracks the amount of data already copied to d_a to avoid redundant transfers. Updated by this function. |
std::strong_ordering) of the substrings up to the comparison limit and the final calculated LCP length (which will not exceed the limit). Returns std::strong_ordering::equal if the substrings match up to the limit, even if the full strings might differ later. d_a, d_largeblock) and a valid CUDA context. other BitString's data is assumed to be accessible directly (e.g., host pinned memory or already on device if applicable to par_find_mismatch). std::bit_ceil and std::countl_zero. Definition at line 587 of file BitString.cuh.
|
inlinenoexcept |
Performs parallel comparison (potentially using CUDA) with another BitString, starting from a given LCP offset.
Compares this BitString with other, assuming the first lcp characters are equal. Compares up to the end of the shorter string using parallel methods if applicable.
| other | The BitString to compare against. | |
| lcp | The starting offset (longest common prefix length) for the comparison, in characters. | |
| d_a | Pointer to device (GPU) memory for this BitString's data. | |
| d_largeblock | Pointer to auxiliary device (GPU) memory. | |
| [in,out] | max_copied | Tracks the amount of data copied to d_a. |
par_k_compare(other, lcp, size(), d_a, d_largeblock, max_copied). Definition at line 236 of file BitString.cuh.
| void BitString< CHAR_T, CHAR_SIZE_BITS >::print_bytes | ( | std::ostream & | os | ) | const |
Prints the raw byte representation of the BitString to an output stream.
Iterates through words and extracts bytes based on WORD_SIZE_BITS. Handles the last partial word correctly.
Writes the packed uintmax_t data to the stream byte by byte, respecting the actual number of characters (size()). Handles partial final words correctly.
| os | The output stream (e.g., std::cout, std::ofstream) to write to. |
Definition at line 669 of file BitString.cuh.
|
noexcept |
Appends a single character to the end of the BitString.
Adds the character to the last word, potentially allocating a new word if needed.
| c | The character to append. |
Definition at line 524 of file BitString.cuh.
|
inlinenoexcept |
Performs sequential comparison with another BitString, starting from a given LCP offset.
Compares this BitString with other, assuming the first lcp characters are equal. Compares up to the end of the shorter string.
| other | The BitString to compare against. |
| lcp | The starting offset (longest common prefix length) for the comparison, in characters. |
seq_k_compare(other, lcp, size()). Definition at line 187 of file BitString.cuh.
|
noexcept |
Performs sequential comparison with another BitString, starting from a given LCP offset, up to a maximum length.
Iterates word by word, finds the first mismatch using XOR and countl_zero.
Compares this BitString with other, assuming the first lcp characters are already known to be equal. The comparison proceeds character by character (packed within words) sequentially, but stops strictly at lcp + max_compare characters or the end of the shorter string, whichever comes first.
| other | The BitString to compare against. |
| lcp | The starting offset (longest common prefix length) for the comparison, in characters. |
| max_compare | The maximum number of additional characters to compare beyond the initial lcp. The comparison will not proceed beyond lcp + max_compare total characters from the start. |
std::strong_ordering) of the substrings up to the comparison limit and the final calculated LCP length (which will not exceed the limit). Returns std::strong_ordering::equal if the substrings match up to the limit, even if the full strings might differ later. std::countl_zero. Definition at line 546 of file BitString.cuh.
|
inlinenoexcept |
Returns the number of characters stored in the BitString.
Definition at line 129 of file BitString.cuh.
| void BitString< CHAR_T, CHAR_SIZE_BITS >::to_file | ( | const std::string & | filename | ) | const |
Writes the BitString contents to a file in binary format.
Opens the file, calls the stream version of to_file, and closes the file.
Serializes the size and the packed data array to the specified file.
| filename | The path to the output file. |
| std::runtime_error | If the file cannot be opened for writing. |
Definition at line 727 of file BitString.cuh.
| void BitString< CHAR_T, CHAR_SIZE_BITS >::to_file | ( | std::ofstream & | file | ) | const |
Writes the BitString contents to an already open output file stream.
Writes the size, then the raw uintmax_t data. Does not perform error checking on writes.
Serializes the size and the packed data array to the stream.
| file | The std::ofstream object, opened in binary mode. |
Definition at line 720 of file BitString.cuh.
|
noexcept |
Converts the BitString back to a standard std::string.
Iterates through characters using iterators and appends them to a string. Assumes CHAR_T is convertible to char.
CHAR_T is convertible to char. Definition at line 775 of file BitString.cuh.
|
inlinenoexcept |
Returns the number of uintmax_t words used for storage.
Definition at line 135 of file BitString.cuh.
|
staticconstexpr |
Number of characters that can be packed into a single storage word.
Definition at line 63 of file BitString.cuh.
|
private |
Internal storage vector holding the packed character data.
Definition at line 250 of file BitString.cuh.
|
staticconstexprprivate |
Compile-time generated array of bit masks.
Definition at line 276 of file BitString.cuh.
|
staticconstexprprivate |
Compile-time generated array of bit shift amounts.
Definition at line 264 of file BitString.cuh.
|
private |
The number of characters stored in the BitString.
Definition at line 252 of file BitString.cuh.
|
staticconstexpr |
Minimum number of characters required to trigger parallel comparison. Derived from MIN_PAR_COMPARE_WORD_SIZE.
Definition at line 65 of file BitString.cuh.
|
staticconstexpr |
Size of the underlying storage word (uintmax_t) in bits.
Definition at line 61 of file BitString.cuh.