The user types in a product, for example, “noodles”. First, we defined and called a function, called “choose_a_product”, with which the user gets asked, which product they would like to choose. To do so, we used the Python Selenium package and a Chromedriver to scrape the needed information from the websites. We looked for all supermarkets for which web scraping works and wrote a script for each of them. Another issue we encountered was that some data sources are quite messy and HTML-files are very different from one another.īecause of all of those obstacles, we had to take a step back and simplify our project. We would have to do it over an external server, which would be too complex. For creating a platform, we would have needed someone with web development skills as a team member. For the location, we would have to work with Google Maps, which would get too complex very quickly and would therefore go beyond the scope of this project. Most websites also do not provide the option of looking for the assortment of a certain region, which might slightly differ from place to place. When we tried to work with REWE, we bumped into some legal issues because of “Captcha”, which is a program that tests if the user is a human or robot. For instance, if you go on the Lidl online shop and search for “bread”, the first result is a toaster. Turned out, only a few supermarkets actually have this kind of online assortment, many of which only display their current discounts or non-edible offers. As so often, the process and final result of our project did not go as planned. To get started, we were looking for supermarkets with online shops or online assortments. This result will be the output the user on the platform gets in the end. In the end, the list gets sorted by total price and distance, and again, the best compromise between those two parameters is found. If the user puts in a whole shopping list, the scraper first gets all the cheapest prices of the products in each supermarket and then puts the sum of those prices, the name of the supermarket, and the distance of it in a Pandas DataFrame. Afterward, the program filters the “price” and “distance” columns and then finds the best compromise between these two parameters. The name and price of the results then get saved into a Pandas DataFrame with columns “product”, “price”, “supermarket” and “distance”. The scraper should then find the cheapest offer of the product in each supermarket close to the user. We planned on getting our data ourselves through web-scraping. Our original plan was to create a platform on which the user can add a single product or a shopping list. Our goal is also to help necessitous people, who do not have enough money to spend a lot on food. This is why we made it our task to develop a program to find the cheapest supermarket for all the groceries you need. It’s a pain to go to several supermarkets to get the cheapest offer of each product or to save money by living off of bread and water. We realized that most variable costs are spent on food and grocery shopping. This data gets filtered to get the cheapest supermarket and the final price as an output.Īs a team consisting of mostly students, all of us know: life is expensive. To do so, the script web-scrapes through the online assortments of several shops to get the needed data which then gets saved into a CSV-file. students and necessitous people to find the cheapest supermarket for their shopping list. This project was carried out as part of the TechLabs “Digital Shaper Program” in Hamburg (winter term 2020/21). Scrimper - Find the cheapest supermarket for your groceries
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |