Digital.gov Guide
Understanding the Site Scanning program
Access the data
Learn how to get started and download data from the Site Scanning program.
Reading time: 2 minutes
Get started
The easiest way to begin accessing and using data from the Site Scanning program is to:
- Download the primary CSV dataset.
- Open the data in a spreadsheet program.
- Apply filters to view the websites for your agency, bureau, or domain only.
- Hide or delete any unused columns to make the spreadsheet faster and more responsive.
- Review the data dictionary to understand the fields for each website.
Then, start looking at the data and using it to generate insights and make data-informed decisions.
Most data fields are straightforward. Review the technical details if you want to dig deeper or pursue more sophisticated tasks.
Download the data
The scan data is exported on a weekly basis. You can download it with live URLs only or all URLs.
Primary dataset with live URLs only
The primary dataset includes scan data for all live URLs only. In other words, it only includes the websites for which Final URL - Live has a value of TRUE. The dataset excludes machine-readable data files such as XML and JSON files.
Full dataset with all URLs
The full dataset includes scan data for all URLs that were scanned, regardless of whether they are live or not. Some URLs may be inaccessible over the public internet, no longer live, or experiencing downtime.
Access the API
The Site Scanning program provides an API that you can use to access all of the scan data. Visit the Site Scanning API page for documentation, including an API key registration.
Contact the Site Scanning team
If you have any questions, please email the Site Scanning team at site-scanning@gsa.gov.
They welcome your feedback, including suggestions for federal websites to add or remove from the Federal Website Index.
