Project Architecture

Architecture

The project is split into four stages in the 'pipeline'.

1. Import - Converts input files (including pdfs, images, and text files) to plain text.
2. Tokenization - Converts plain text to a list of words.
3. Indexing - Inserts information about associations between files and the words they contain.
4. Maintainance - Detecting file updates and updating the database accordingly. Also allows for removing a file from the index.