FanFiction Search Engine: A full-text search engine for Code Lyoko FanFiction. Final project for Search Informatics.
- Source Code: https://github.com/hayesall/CLFanfictionSearchEngine
The final portion of the “Search Informatics” course I took involved developing an entire search engine—including a frontend website, a backend for returning pages, the spiders for crawing the web, and scrapers for turning pages into searchable text.
At the time, FanFiction.Net lacked a full-text search option. It was possible to search for content in the “title” or “synopsis” sections, but there was no way to know if any of the stories mentioned “twitter.”
The PageRank algorithm operates on a directed graph, so it was possible to incorporate some knowledge about how the website worked. There are “Users” and “Stories,” which I generally drew with violet or blue nodes. Users can write or review stories—therefore I could crawl over all the stories, extract text to build the search engine, and record user interactions to build the network.
When visualized, the inner region appeared to be extremely dense. There were some users and stories with less attention, but the inner region was quite dense and showed a high amount of engagement between users and stories.
I wrote some notes about this as a blog post: Network of Code Lyoko FanFiction.
This search engine is no longer maintained. I still study networks and learning on arbitrarily-structured data though.