• 0 Posts
  • 4 Comments
Joined 1 year ago
cake
Cake day: June 11th, 2023

help-circle


  • The search engine market isn’t quite as diverse as it may appear https://www.searchenginemap.com/

    There are maybe 4 or so ‘crawlers’, and the rest buys access to the part of their data they are willing to sell to others.

    Running a crawler with the current size and complexity of the internet is expensive, and complicated. Then there is sifting and sorting the data in a reasonable searchable format, and then there is the quality problem, etc.

    Much easier to license data access from a provider (Usually Bing or Google or both) and just offer some added features on top, like no tracking, different result UI, custom filtering values per Bing or Google’s APIs that make your own “secret sauce”, etc.


  • Can someone explain to me what’s the point in having a lot of small instances of something like Lemmy?

    I’m very familiar with Azure, and looking at the docker-compose file and AWS setup, it’s very straightforward to setup a simple instance on Azure container apps. How much it costs you will highly depend on what you want to do with it and how you expect it to be used.

    Like, how much traffic are you expecting?