genetics Namespace Reference

Contains utilities for working with genetic data and file formats. More...

Classes
struct	GeneManager
	Manages efficient loading of multiple genetic sequences from a combined file. More...

Enumerations
enum class	FileType {   GBK , FNA , DNA , DNA_BIN ,   DNA_BIN_COMBINED , UNKNOWN }
	Enumerates the supported genetic file formats. More...

Functions
Gene	from_gbk (const std::string &filename)
	Loads a gene from a GenBank format file.
Gene	from_fna (const std::string &filename)
	Loads a gene from a FASTA nucleotide format file.
Gene	from_dna (const std::string &filename)
	Loads a gene from a plain text DNA format file.
Gene	from_file (const std::string &filename)
	Loads a gene from a file with automatic format detection.
Gene	from_file (const std::filesystem::path &path)
	Loads a gene from a file path with automatic format detection.
Gene	from_file (const std::filesystem::directory_entry &entry)
	Loads a gene from a directory entry with automatic format detection.
void	to_dna (const Gene &gene, const std::string &filename)
	Saves a gene to a plain text DNA format file.
std::vector< Gene >	from_directory (const std::string &dirname)
	Loads all genes from files in a directory with automatic format detection.
std::vector< Gene >	from_directory (const std::string &dirname, FileType type)
	Loads all genes from files of a specific format in a directory.
std::vector< std::string >	get_dna_strings_from_directory (const std::string &dirname)
	Extracts DNA strings from all supported files in a directory.
FileType	get_file_type (const std::string &filename)
	Determines the file type based on its filename extension.
FileType	get_file_type (const std::filesystem::path &path)
	Determines the file type based on its path's extension.
FileType	get_file_type (const std::filesystem::directory_entry &entry)
	Determines the file type based on a directory entry's path extension.
Gene	from_stream (std::istream &stream)
	Loads a gene from an input stream.
std::vector< std::filesystem::path >	get_virus_nucleotide_paths ()
	Gets paths to all virus nucleotide files in the virus directory.
void	combine_dna_bin_files (const std::string &dirname, const std::string &combined_filename)
	Combines multiple binary DNA files into a single combined file.

Variables
static const std::string	GENETIC_DIRECTORY = "genetic/"
	Path to the genetic data directory.
static const std::string	VIRUS_DIRECTORY = GENETIC_DIRECTORY + "virus/"
	Path to the virus genome data directory.
static const std::string	ABC_HUMI_DIRECTORY = GENETIC_DIRECTORY + "ABC-HuMi/"
	Path to the ABC-HuMi dataset directory.
static const std::string	DNA_EXTENSION = ".dna"
	File extension for plain text DNA files.
static const std::string	DNA_BIN_EXTENSION = ".dna.bin"
	File extension for binary DNA files.
static const std::string	DNA_BIN_COMBINED_EXTENSION = ".dna.bin.combined"
	File extension for combined binary DNA files.
static const std::string	DNA_BIN_SIZES_EXTENSION = ".dna.bin.sizes"
	File extension for binary DNA file size information.
static const std::string	GBK_EXTENSION = ".gbk"
	File extension for GenBank format files.
static const std::string	FNA_EXTENSION = ".fna"
	File extension for FASTA nucleotide format files.

Detailed Description

Contains utilities for working with genetic data and file formats.

Provides constants, enums, and functions for loading, saving, and manipulating genetic data from various file formats. The namespace encapsulates all functionality related to genetic sequence processing.

Enumeration Type Documentation

FileType

enum class genetics::FileType

strong

Enumerates the supported genetic file formats.

This enumeration identifies the different file formats that can be processed by the genetics utility functions. Each format has specific parsing rules implemented in the corresponding from_* functions.

Enumerator
GBK	GenBank format (.gbk)
FNA	FASTA nucleotide format (.fna)
DNA	Plain text DNA format (.dna)
DNA_BIN	Binary DNA format (.dna.bin)
DNA_BIN_COMBINED	Combined binary DNA format (.dna.bin.combined)
UNKNOWN	Unknown or unsupported file format.

Definition at line 73 of file genetics.cuh.

    {
        GBK,            
        FNA,            
        DNA,            
        DNA_BIN,        
        DNA_BIN_COMBINED, 
        UNKNOWN         
    };

Function Documentation

combine_dna_bin_files()

void genetics::combine_dna_bin_files	(	const std::string &	dirname,
		const std::string &	combined_filename
	)

Combines multiple binary DNA files into a single combined file.

Creates a combined binary file and a separate size file that tracks the size of each original file for later extraction.

Parameters

dirname	Directory containing binary DNA files to combine.
combined_filename	Name of the output combined file.

Definition at line 335 of file genetics.cu.

{
    // Phase 1: Collect paths of all DNA_BIN files in the directory
    WallTimer timer;
    timer.start("Adding filepaths to vector");
    std::vector<std::filesystem::path> filepaths;
    for (const auto& entry : std::filesystem::directory_iterator(dirname))
    {
        if (get_file_type(entry) == FileType::DNA_BIN)
        {
            filepaths.emplace_back(entry.path());
        }
    }
    timer.print();
 
    printf("Added %lu filepaths to vector\n", filepaths.size());
 
    // Phase 2: Create the combined file header with file count, names, and placeholders for positions
    timer.start("Writing names to binary file");
    std::ofstream file(dirname + combined_filename + DNA_BIN_COMBINED_EXTENSION, std::ios::binary);
    if (!file.is_open())
    {
        throw std::invalid_argument("File not found");
    }
 
    // Write the number of files first for easy parsing later
    size_t num_names = filepaths.size();
    file.write(reinterpret_cast<const char*>(&num_names), sizeof(num_names));
 
    // For each file, write its name and reserve space for its position in the file
    std::vector<std::streampos> streampos_positions;
    for (const auto& filepath : filepaths)
    {
        // Extract the base filename without the .dna.bin extension
        auto name = filepath.filename().string();
        name = name.substr(0, name.size() - DNA_BIN_EXTENSION.size());
 
        // Write the name with null terminator
        file.write(name.c_str(), name.size());
        file.write("\0", 1);
        
        // Store the current position to update later with the actual gene data position
        streampos_positions.push_back(file.tellp());
        
        // Write a placeholder for the position (will update later)
        file.write(reinterpret_cast<const char*>(&streampos_positions.back()), sizeof(streampos_positions.back()));
    }
 
    timer.print();
 
    // Phase 3: Write each gene to the file and update its position in the header
    timer.start();
    for (unsigned i = 0; i < filepaths.size(); ++i)
    {
        // Remember current position where gene data will start
        auto pos = file.tellp();
        
        // Go back to the position placeholder in the header and update it
        file.seekp(streampos_positions[i]);
        file.write(reinterpret_cast<const char*>(&pos), sizeof(pos));
        
        // Return to the end of the file to write the gene data
        file.seekp(pos);
 
        // Load and write the gene
        auto gene = Gene::from_file(filepaths[i].string());
        gene.to_file(file);
 
        // Progress reporting for long operations
        if (i % 1000 == 0)
        {
            printf("Wrote %u files, took %s\n", i, pretty_print(timer.elapsed_nanoseconds()).c_str());
        }
    }
}

from_directory() [1/2]

std::vector< Gene > genetics::from_directory

(

const std::string &

dirname

)

Loads all genes from files in a directory with automatic format detection.

Implementation of from_directory function that loads genes from all files in a directory.

Parameters

dirname

Path to the directory containing genetic data files.

Returns

std::vector<Gene> Vector of parsed genetic sequences.

See also

from_directory(const std::string&, FileType)

Iterates through all files in the specified directory and attempts to load each file as a genetic sequence using automatic format detection. This version handles all supported file types.

Parameters

dirname

Path to the directory containing genetic data files.

Returns

std::vector<Gene> Vector of parsed genetic sequences.

Definition at line 183 of file genetics.cu.

{
    std::vector<Gene> genes;
 
    for (const auto& entry : std::filesystem::directory_iterator(dirname))
    {
        genes.emplace_back(from_file(entry));
    }
 
    return genes;
}

from_directory() [2/2]

std::vector< Gene > genetics::from_directory	(	const std::string &	dirname,
		FileType	type
	)

Loads all genes from files of a specific format in a directory.

Parameters

dirname	Path to the directory containing genetic data files.
type	The specific file type to load.

Returns

std::vector<Gene> Vector of parsed genetic sequences.

See also

from_file

Definition at line 195 of file genetics.cu.

{
    std::vector<Gene> genes;
 
    // Iterate through directory and filter by the requested file type
    for (const auto& entry : std::filesystem::directory_iterator(dirname))
    {
        // Only process files matching the specified type
        if (get_file_type(entry) == type)
        {
            genes.emplace_back(from_file(entry));
        }
    }
 
    return genes;
}

from_dna()

Gene genetics::from_dna

(

const std::string &

filename

)

Loads a gene from a plain text DNA format file.

Implementation of from_dna function for loading genes from DNA format files.

Parameters

filename

Path to the DNA (.dna) file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

See also

from_file

Opens a DNA format file and passes the file stream to from_stream to extract the nucleotide sequence. DNA format files contain only nucleotide characters.

Parameters

filename

Path to the DNA format file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

Definition at line 152 of file genetics.cu.

{
    std::ifstream file(filename);
    if (!file.is_open())
    {
        throw std::invalid_argument("File not found");
    }
 
    Gene gene;
 
    std::string line;
    while (std::getline(file, line))
    {
        for (char c : line)
        {
            gene.push_back(char_to_nucleotide[c]);
        }
    }
 
    return gene;
}

from_file() [1/3]

Gene genetics::from_file

(

const std::filesystem::directory_entry &

entry

)

Loads a gene from a directory entry with automatic format detection.

Parameters

entry

A filesystem directory entry pointing to the genetic data file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened or has an unsupported format.

See also

from_file(const std::filesystem::path&)

Definition at line 318 of file genetics.cu.

{
    return from_file(entry.path().string());
}

from_file() [2/3]

Gene genetics::from_file

(

const std::filesystem::path &

path

)

Loads a gene from a file path with automatic format detection.

Parameters

path	A filesystem path to the genetic data file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened or has an unsupported format.

See also

from_file(const std::string&)

Definition at line 313 of file genetics.cu.

{
    return from_file(path.string());
}

from_file() [3/3]

Gene genetics::from_file

(

const std::string &

filename

)

Loads a gene from a file with automatic format detection.

Determines the file type based on its extension and calls the appropriate parsing function.

Parameters

filename

Path to the genetic data file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened or has an unsupported format.

See also

get_file_type

Definition at line 284 of file genetics.cu.

{
    FileType type = get_file_type(filename);
 
    switch (type)
    {
    case FileType::GBK:
        return from_gbk(filename);
    case FileType::FNA:
        return from_fna(filename);
    case FileType::DNA:
        return from_dna(filename);
    case FileType::DNA_BIN:
        return Gene::from_file(filename);
    default:
        throw std::invalid_argument("Unknown file type");
    }
}

from_fna()

Gene genetics::from_fna

(

const std::string &

filename

)

Loads a gene from a FASTA nucleotide format file.

Implementation of from_fna function for parsing FASTA nucleotide format files.

Parameters

filename

Path to the FASTA nucleotide (.fna) file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

See also

from_file

FASTA files start with a header line (beginning with '>') followed by the nucleotide sequence. This function skips the first line (header) and then passes the file stream to from_stream to extract the actual nucleotide sequence.

Parameters

filename

Path to the FASTA nucleotide file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

Definition at line 107 of file genetics.cu.

{
    std::ifstream file(filename);
    if (!file.is_open())
    {
        throw std::invalid_argument("File not found");
    }
 
    // step 1: skip header (just one line)
 
    std::string line;
    std::getline(file, line);
 
    return from_stream(file);
}

from_gbk()

Gene genetics::from_gbk

(

const std::string &

filename

)

Loads a gene from a GenBank format file.

Implementation of from_gbk function for parsing GenBank format files.

Parameters

filename

Path to the GenBank (.gbk) file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

See also

from_file

GenBank files have a specific format with a header section followed by a section that begins with "ORIGIN" followed by the nucleotide sequence. This function skips the header section and then passes the file stream to from_stream to extract the actual nucleotide sequence.

Parameters

filename

Path to the GenBank file.

Returns

Gene The parsed genetic sequence.

Exceptions

std::invalid_argument

if the file cannot be opened.

Definition at line 71 of file genetics.cu.

{
    std::ifstream file(filename);
    if (!file.is_open())
    {
        throw std::invalid_argument("File not found");
    }
 
    // step 1: skip header
    // keep reading until the line just read is "ORIGIN"
 
    std::string line;
    while (std::getline(file, line))
    {
        if (line == "ORIGIN")
        {
            break;
        }
    }
 
    // step 2: read nucleotides
    // read line by line, character by character, discarding any non-nucleotide characters
 
    return from_stream(file);
}

from_stream()

Gene genetics::from_stream

(

std::istream &

stream

)

Loads a gene from an input stream.

Implementation of from_stream function that parses genetic sequences from a stream.

Reads valid nucleotides from the stream until EOF, skipping invalid characters.

Parameters

stream

The input stream to read from.

Returns

Gene The parsed genetic sequence.

Reads the stream line by line, extracting valid nucleotides and adding them to the gene sequence. Invalid characters are skipped. Aborts if a FASTA header marker ('>') is encountered, as this indicates incorrect parsing logic.

Parameters

stream

Input stream containing genetic data.

Returns

Gene The parsed genetic sequence.

Definition at line 36 of file genetics.cu.

{
    Gene gene;
 
    std::string line;
    while (std::getline(stream, line))
    {
        for (char c : line)
        {
            if (is_valid_nucleotide(c))
            {
                gene.push_back(char_to_nucleotide[c]);
            }
            if (c == '>')
            {
                printf("NOT GOOD\n");
                exit(0);
            }
        }
    }
 
    return gene;
}

get_dna_strings_from_directory()

std::vector< std::string > genetics::get_dna_strings_from_directory

(

const std::string &

dirname

)

Extracts DNA strings from all supported files in a directory.

Parameters

dirname

Path to the directory containing genetic data files.

Returns

std::vector<std::string> Vector of DNA strings.

Definition at line 212 of file genetics.cu.

{
    std::vector<std::string> dna_strings;
 
    // Iterate through all files in the directory
    for (const auto& entry : std::filesystem::directory_iterator(dirname))
    {
        // Process only DNA format files
        if (get_file_type(entry) == FileType::DNA)
        {
            std::ifstream file(entry.path());
            if (!file.is_open())
            {
                throw std::invalid_argument("File not found");
            }
 
            // Read just the first line from each file
            // Note: This assumes DNA files contain a single line with the entire sequence
            std::string dna_string;
            std::getline(file, dna_string);
            dna_strings.emplace_back(dna_string);
        }
    }
 
    return dna_strings;
}

get_file_type() [1/3]

FileType genetics::get_file_type

(

const std::filesystem::directory_entry &

entry

)

Determines the file type based on a directory entry's path extension.

Parameters

entry

A filesystem directory entry to analyze.

Returns

FileType The determined file type, or FileType::UNKNOWN if not recognized.

See also

get_file_type(const std::filesystem::path&)

Definition at line 308 of file genetics.cu.

{
    return get_file_type(entry.path().string());
}

get_file_type() [2/3]

FileType genetics::get_file_type

(

const std::filesystem::path &

path

)

Determines the file type based on its path's extension.

Parameters

path	A filesystem path to analyze.

Returns

FileType The determined file type, or FileType::UNKNOWN if not recognized.

See also

get_file_type(const std::string&)

Definition at line 303 of file genetics.cu.

{
    return get_file_type(path.string());
}

get_file_type() [3/3]

FileType genetics::get_file_type

(

const std::string &

filename

)

Determines the file type based on its filename extension.

Parameters

filename

The filename to analyze.

Returns

FileType The determined file type, or FileType::UNKNOWN if not recognized.

Definition at line 256 of file genetics.cu.

{
    if (string_ends_with(filename, GBK_EXTENSION))
    {
        return FileType::GBK;
    }
    else if (string_ends_with(filename, FNA_EXTENSION))
    {
        return FileType::FNA;
    }
    else if (string_ends_with(filename, DNA_EXTENSION))
    {
        return FileType::DNA;
    }
    else if (string_ends_with(filename, DNA_BIN_EXTENSION))
    {
        return FileType::DNA_BIN;
    }
    else if (string_ends_with(filename, DNA_BIN_COMBINED_EXTENSION))
    {
        return FileType::DNA_BIN_COMBINED;
    }
    else
    {
        return FileType::UNKNOWN;
    }
}

get_virus_nucleotide_paths()

std::vector< std::filesystem::path > genetics::get_virus_nucleotide_paths

(

)

Gets paths to all virus nucleotide files in the virus directory.

Returns

std::vector<std::filesystem::path> Vector of paths to virus nucleotide files.

Definition at line 323 of file genetics.cu.

{
    std::vector<std::filesystem::path> paths;
 
    for (const auto& entry : std::filesystem::directory_iterator(VIRUS_DIRECTORY))
    {
        paths.emplace_back(entry.path());
    }
 
    return paths;
}

to_dna()

void genetics::to_dna	(	const Gene &	gene,
		const std::string &	filename
	)

Saves a gene to a plain text DNA format file.

Implementation of to_dna function for saving genes to DNA format files.

Parameters

gene	The genetic sequence to save.
filename	Path where the DNA file will be saved.

Exceptions

std::runtime_error

if the file cannot be created or written to.

Writes the gene to a text file in DNA format, using the stream output operator of the Gene class to convert the gene to a string representation.

Parameters

gene	The genetic sequence to save.
filename	Path where the DNA file will be saved.

Exceptions

std::invalid_argument

if the file cannot be opened for writing.

Definition at line 132 of file genetics.cu.

{
    std::ofstream file(filename);
    if (!file.is_open())
    {
        throw std::invalid_argument("File not found");
    }
 
    file << gene;
}

Variable Documentation

ABC_HUMI_DIRECTORY

const std::string genetics::ABC_HUMI_DIRECTORY = GENETIC_DIRECTORY + "ABC-HuMi/"

static

Path to the ABC-HuMi dataset directory.

Definition at line 46 of file genetics.cuh.

DNA_BIN_COMBINED_EXTENSION

const std::string genetics::DNA_BIN_COMBINED_EXTENSION = ".dna.bin.combined"

static

File extension for combined binary DNA files.

Definition at line 55 of file genetics.cuh.

DNA_BIN_EXTENSION

const std::string genetics::DNA_BIN_EXTENSION = ".dna.bin"

static

File extension for binary DNA files.

Definition at line 52 of file genetics.cuh.

DNA_BIN_SIZES_EXTENSION

const std::string genetics::DNA_BIN_SIZES_EXTENSION = ".dna.bin.sizes"

static

File extension for binary DNA file size information.

Definition at line 58 of file genetics.cuh.

DNA_EXTENSION

const std::string genetics::DNA_EXTENSION = ".dna"

static

File extension for plain text DNA files.

Definition at line 49 of file genetics.cuh.

FNA_EXTENSION

const std::string genetics::FNA_EXTENSION = ".fna"

static

File extension for FASTA nucleotide format files.

Definition at line 64 of file genetics.cuh.

GBK_EXTENSION

const std::string genetics::GBK_EXTENSION = ".gbk"

static

File extension for GenBank format files.

Definition at line 61 of file genetics.cuh.

GENETIC_DIRECTORY

const std::string genetics::GENETIC_DIRECTORY = "genetic/"

static

Path to the genetic data directory.

Definition at line 40 of file genetics.cuh.

VIRUS_DIRECTORY

const std::string genetics::VIRUS_DIRECTORY = GENETIC_DIRECTORY + "virus/"

static

Path to the virus genome data directory.

Definition at line 43 of file genetics.cuh.