TwitterScraping

Twitter Scraping Project

Problem Statement

Today, data is scattered everywhere in the world. Especially in social media, there may be a big quantity of data on Facebook, Instagram, Youtube, Twitter, etc. This consists of pictures and films on Youtube and Instagram as compared to Facebook and Twitter. To get the real facts on Twitter, you want to scrape the data from Twitter. You Need to Scrape the data like (date, id, url, tweet content, user,reply count, retweet count,language, source, like count etc) from twitter.

Directory Structure

This repository consists the file required to create Twitter Scrapper Application

The “main.py” file consists of major code required to develop the applcation. Pycharm IDE is recommended.

The “lottie.py” file has code required for animation of image in Frontend Streamlit User Interface.

Requirements

Python Scripting

Streamlit for Front End UI

Snscrape library to extract data from Twitter

Pandas to create Data Frames

Mongodb to store data in database

Features with Screenshot

This is how the Front end of UI looks like

After entering the Twitter Hashtag, It wil ask to insert the number of hashtag to search from start date to end date.

This is how the extracted data from twitter looks like with all the required data columns. Here we can upload the data to database, download the data in csv or json format