I created my own search engine

schBenedikt - Jun 9 - - Dev Community

I have created my own search engine and I'll tell you how it works.

First of all, I created my own web crawler. It runs on Python and stores various meta data in a mysql database.

GitHub logo SchBenedikt / web-crawler

A simple web crawler using Python that stores the metadata of each web page in a database.

web-crawler

A simple web crawler using Python that stores the metadata of each web page in a database.




The search engine, which I created with Bootstrap, then retrieves the results from mySQL. The bottom right always shows how long the query took.

GitHub logo SchBenedikt / search-engine

The matching search engine to my web crawler.




Please note that no robots.txt files are currently taken into account, which is why you cannot simply crawl every page.

What do you think of the project?

Feel free to write it in the comments!

Image description

Image description

Image description

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .