Is Web Scraping Legal? What the Law Actually Looks At

The short version

So the honest answer to "is web scraping legal?" is "it depends on five things". The rest of this article goes through them.

Five questions that decide most cases

1. Is the page public, or behind a login?

The main U.S. anti-hacking law, the Computer Fraud and Abuse Act (CFAA), punishes access to a computer "without authorization". In Van Buren v. United States (2021) the Supreme Court read the law narrowly: it is about entering parts of a system that are off limits to you, not about misusing access you legitimately have.

Applying that reasoning, the Ninth Circuit held in hiQ Labs v. LinkedIn (April 2022) that pages anyone can view without logging in have no "gate" to break through, so scraping them is unlikely to be unauthorized access under the CFAA. Pages behind a login are different: you got in by accepting terms, and those terms usually say something about automated collection.

2. Does the data describe people?

Names, email addresses, photos, profiles and anything else that identifies a person are personal data. Under the GDPR it makes no difference that the person published it themselves: you still need a legal basis to collect and use it. Regulators have acted on this. Clearview AI, which built a face database from scraped photos, was fined €20 million by France's CNIL in 2022 and €30.5 million by the Dutch data protection authority in 2024.

If you do not need personal data, do not collect it. Leaving those columns out is the simplest compliance step there is.

3. Is the content protected by copyright or database rights?

Facts are free. In Feist v. Rural (1991) the U.S. Supreme Court held that a list of facts gets no copyright just because collecting it took effort. A price, a product specification or an opening time is a fact. An article, a product photo or a written review is creative work, and copying it wholesale is a different matter.

The EU adds a layer: the Database Directive gives the maker of a database a separate right against extraction of substantial parts of it, even when the individual entries are plain facts.

4. What do the site's terms say?

Terms of use are a contract question, separate from hacking law. The hiQ case shows both sides: after winning on the CFAA, hiQ lost on contract. In November 2022 the district court found it had breached LinkedIn's user agreement, and in December 2022 the case closed with a consent judgment of $500,000 against it.

Whether terms bind you depends on whether you accepted them and on what they actually say. In Meta v. Bright Data (January 2024) a California federal court found that Meta's terms, as written, did not prohibit scraping public pages while logged out. In X Corp. v. Bright Data (May 2024) another judge dismissed X's claims, noting that users, not X, own what they post. In Europe, the Court of Justice held in Ryanair v PR Aviation (2015) that a site whose data is not protected by database law may still restrict its use by contract.

5. How are you collecting it?

Sending thousands of requests a minute can slow a site down for everyone else, and getting around blocks — rotating addresses after being banned, solving CAPTCHAs automatically, ignoring a cease-and-desist letter — turns a neutral activity into one a court will look at unkindly. A robots.txt file is a convention rather than a law, but ignoring it is the kind of fact that gets quoted in complaints.

The rulings in one table

Lower risk and higher risk

A checklist before you start

1. Read the site's terms of use and its robots.txt. Look for an official export or API first; it is often easier as well as safer.
2. Decide which columns you actually need and leave out anything that identifies a person.
3. Stay on pages that are public, or that you are entitled to see.
4. Keep the pace gentle and stop if the site asks you to.
5. Use the data for analysis. Do not republish text, images or large parts of a database.
6. If the project is commercial or large, get legal advice before, not after.

Where a browser extension fits

A scraper that runs in your own browser reads pages you have open, about as fast as a person clicking through them. That keeps it away from the heavy-load end of the scale. It does not answer the first four questions for you: what you collect and what you do with it remain your decision. Easy Scraper AI works this way, and its terms say the same thing in plainer words.

If you want to see what a careful, small-scale collection looks like in practice, the price scraping guide walks through one on a practice site, and the guide on how to scrape multiple pages covers pacing. Shorter answers to common worries are in the frequently asked questions.

Sources

• Van Buren v. United States, U.S. Supreme Court, June 2021
• hiQ Labs v. LinkedIn, Ninth Circuit, April 18, 2022; settlement and consent judgment, December 2022
• Meta v. Bright Data, summary judgment of January 23, 2024
• X Corp. v. Bright Data, dismissal of May 2024
• Feist Publications v. Rural Telephone Service, U.S. Supreme Court, 1991
• Ryanair v PR Aviation, C-30/14, EU Court of Justice, January 2015
• General Data Protection Regulation and Database Directive 96/9/EC
• Clearview AI decisions: CNIL, 2022 and Dutch DPA, 2024