Ayush P Gupta

Jul 19, 2020

5 min read

DATA SCRAPING

Website Scraping with Dart (Flutter)

How to extract data from website using Flutter

What is a Website Scraping?

Website scraping, or web data extraction, is the process of extracting or scraping data from websites.

Gather Fuel and Gear Up

1. Create a new Flutter Project

You need a flutter project to get started, although you can create a standalone Dart console app but to make things easy we’ll do this in an app project.
Open up terminal and run following command.

flutter create flutter_scrap

2. Add web_scraper dependency

There is a pub package named web_scraper which make scraping easy.
Open up pubspec.yaml and add following line in dependencies section

dependencies:
web_scraper: ^0.0.6

3. Gather website and data to scrap

For this article, we will be using unacademy.com website.

Extract titles of chapters

4. See what’s in HTML

A web scraper simply reads and parses the website’s HTML and extracts the different elements (like <div>,<a>) from it. Now one can provide condition to whichever elements he likes to scrap.

A chapter title’s element

5. Write Scraper

Have a look at documentation of web_sraper. There are 3 parts:

  1. Domain — https://unacademy.com
  2. Endpoint — /course/gravitation-for-iit-jee/D5A8YSAJ
  3. Address of element to be extracted
'div.Week__Wrapper-sc-1qeje5a-2 > a.Link__StyledAnchor-sc-1n9f3wx-0 > div.ItemCard__ItemInfo-xrh60s-1 > h6.H6-sc-1gn2suh-0'
  1. div.Week__Wrapper-sc-1qeje5a-2
  2. a.Link__StyledAnchor-sc-1n9f3wx-0
  3. div.ItemCard__ItemInfo-xrh60s-1
  4. h6.H6-sc-1gn2suh-0
flutter: [Newton’s Law of Gravitation, Variation in Value of ‘g’: Part 2, Gravitational Field Due to a Point Mass, Variation in Value of ‘g’: Part 1, Gauss Theorem, Gravitational Field due to a Uniform Solid Sphere and Uniform Spherical Shell, Gravitational Potential, Relation Between Gravitational Field and Potential, Gravitational Potential Energy, Binding Energy, Motion of Satellites, Maximum Height attained by a Particle, Trajectory of Satellites, Quality Numerical 001 : Sphere with a Cavity, Quality Numerical 002 : Gravitational Force by a Rod, Quality Numerical 003 : Ring and Sphere, Quality Numerical 004 : Gravitational Field of Semi Circular Wire, Quality Numerical 005 : Binding Energy and Escape Velocity, Quality Numerical 006 : Tunnel and Velocity, Quality Numerical 007 : Tunnel and Acceleration, Quality Numerical 008 : Tunnel and Amplitude, Quality Numerical 009 : Gravity Variation, Quality Numerical 010 : Gravity Variation, Quality Numerical 011 : Weightlessness, Quality Numerical 012 : Planet revolving, Quality Numerical 013 : Escape Velocity, Quality Numerical 014 : Angular Momentum, Quality Numerical 015 : Gravitational Field and Potential]
Output of above code

Conclusion

Web scraping is quite a large and complex topic to summarize all in one article. The above presents a very simple use case in real world.

Further Reading