Open data and emerging technologies—including artificial intelligence and distributed ledgers, such as blockchain—hold vast potential to transform public services held back by bureaucracy and outdated IT systems. We are opening the doors to bold, fresh ideas for government accountability, transparency and citizen participation by working with U.S. businesses, civil society groups and others to shape national goals for emerging technologies and open data in public services. At our upcoming collaborative workshop, Emerging Technology and Open Data for a More Open Government, we invite new partners to help craft potential goals to be integrated into the fourth U.
CDC’s surveillance systems track HIV, AIDS, viral hepatitis, STDs, and TB. Getting this information to those who need it most in an accessible, usable, and meaningful format is a primary goal for CDC. The launch of Atlas in 2012 made this a reality with an online resource that gave users tools to create customized tables, maps, and other graphics with the most current CDC surveillance data. Now, we have launched CDC’s NCHHSTP AtlasPlus.
Forbes magazine recently ran an article showcasing six handy mobile apps that were built using federal government open data. The apps range from the Alternative Fueling Station Locator to ZocDoc (a doctor locator). What I especially like about the Forbes article is that the author describes the federal government data sets behind each app. There are many more mobile apps built by federal government agencies or using federal government data sources.
Last [month], NASA Open Innovation Program Manager Dr. Beth Beck and her team traveled to the Canadian Space Agency (CSA) near Montreal, Canada to attend the Inspiring Data Forum graciously hosted by our Open Data neighbors to the North. The goal of this gathering was to bolster the working relationship between the two Space Agency’s Open Data efforts and to present techniques NASA is doing in Open Innovation. The event was heavily attended by CSA employees and also had participants from National Research Council of Canada, Treasury Board Secretariat, Natural Resources Canada, Agricultural and Agri-Food Canada, MaxQ and Social Sciences & Humanities Research Council of Canada.
DigitalGov University (DGU), the events platform for DigitalGov, provides programming to build and accelerate digital capacity by providing webinars and in-person events highlighting innovations, case studies, tools, and resources. Thanks to your participation, DGU hosted over 90 events with 6,648 attendees from over 100 agencies across federal, tribal, state, and local governments. DGU strives to provide training throughout the year that is useful and relevant to you. One of the most resounding comments from digital managers last year was people wanted to be able to attend all of our classes virtually.
Summary: The White House is hosting its first-ever Electric Vehicle Datathon, and nominations for participation are now open. Don’t miss your chance to join the discussion! The White House Domestic Policy Council and Office of Science and Technology Policy will convene its first-ever Electric Vehicle (EV) Datathon on November 29. This event, held in partnership with the Department of Energy and four of its National Laboratories will bring together EV experts, charging-station providers, cities and states, automakers, and the software-development and data-analysis communities.
Improving the way the government delivers information technology (IT) solutions to its customers isn’t just a goal, it’s our mission. We at the U.S. Patent and Trademark Office know that by publishing our open source code, the public can help us come up with new and better IT solutions. In advance of the new Federal Source Code Policy and in support of the Administration’s Open Government Initiative, we have been publishing content on GitHub for over a year, and it now includes source code for a mobile application for trademarks.
Daily imagery data taken by the Earth Polychromatic Imaging Camera is now accessible via a RESTful API available from the NASA API Portal. The Earth Polychromatic Imaging Camera (EPIC) is an instrument aboard NOAA’s DSCOVR satellite, which orbits at Earth’s Lagrange point, the sweet spot in space where the gravitational tug of the Earth and the Sun is equal. This allows DSCOVR to maintain a stable position between the Earth and Sun and thus a continuous view of the sunlit side of Earth.
In December, I plan to write two postings detailing a scenario analysis for the next ten years of the Federal government’s data technologies. Governments are on the cusp of amazing technological advances propelled by artificial intelligence, blockchain technologies, and the Internet of Things. Also, governments will face new challenges such as the recent global cyber attack that took down Twitter and Netflix. I want to invite you, the reader, to also send in your predictions for the future of Federal government data.
Federal agencies confront tough problems every day. In searching for solutions, agencies will want to attract different perspectives, test new products, build capacity and communities, and increase public awareness. How do they do it? The answer: open innovation. Federal agencies need to engage and collaborate with all sectors of society, a task made easier by online technologies, says a Government Accountability Office (GAO) report issued last week. OPEN INNOVATION: Practices to Engage Citizens and Effectively Implement Federal Initiatives is accompanied by an infographic and podcast, all well worth your while.
The Data Briefing: The Federal Data Cabinet—Promoting Data Literacy, Cultural Change, and the Federal Data Applications Ecosystem
Last Wednesday, the White House held the first Open Data Summit to showcase the open data accomplishments of the Obama Administration. One of the highlights was the formation of a government-wide “data cabinet.” Announced by Chief Data Scientist DJ Patil, the data cabinet is essentially a community of practice comprising the Federal agency’s data professionals. As Dr. Patil explains, the real issues concerning technical projects revolve around cultural issues. I couldn’t agree more.
Here is the outline for our 2016 Open Government Plan. Let us know what you think. We’ve also posted this on GitHub/NASA for your comments: https://github.com/nasa/Open-Gov-Plan-v4. NASA and Open Government NASA is an open government agency based on the founding legislation in the National Aeronautics and Space Act of 1958, which calls for participation and sharing in the conduct of how we go about the business of expanding the frontiers of knowledge, advancing understanding of the universe, and serving the American public.
The United Kingdom’s (UK) Digital Service has researched ways to increase data science literacy among the UK public service. Data science literacy goes further than data literacy, in that civil servants will know how to apply data science concepts and methods in their everyday work. I thought it would be useful to share the UK Digital Service’s findings to help federal government employees develop their data science literacy. Before discussing how to increase the data science skills of federal employees, let’s discuss why.
The Data Briefing: Harnessing the Internet of Things and Synthetic Data to Provide Better Flood Warnings and Prevent Veterans Suicides
Two significant items in federal government data in the last few weeks: The Department of Commerce releases the National Water Model. The National Water Model provides a comprehensive model of river flows so local communities can better prepare for possible flooding events. What is especially amazing about the National Water Model is that it pulls data from over 8,000 stream gauges. Stream gauges are automated measuring stations that measure water flow, height, surface runoff, and other hydrological data.
This week marks a special anniversary for GSA as we celebrate our Data-to-Decision (D2D) platform since its launch in the fall of 2015. D2D is GSA’s data management platform that collects, manages, and analyzes complex data to enable data-driven decision-making. Over the past year, GSA has made major strides forward in understanding how accurate and insightful data can help us be a more data-driven organization. I am proud to say that D2D now has more than 1,000 users and over 100 published dashboards across GSA!
The Data Briefing: FINDing Great Global Development Data Visualizations Courtesy of the State Department
Federal agencies have been releasing some fascinating data visualization tools in the last year. Recently, the State Department unveiled the Beta version of FIND or the “F Interagency Network Databank.” From the description in the FAQ: “The F Interagency Network Databank (FIND) is an online tool that enables users to explore and analyze national level data, and then share what they discover. FIND was designed around the needs of U.
Summary: Today, we’re releasing the Federal Source Code policy to support improved access to custom software code developed by or for the Federal Government. “If we can reconceive of our government so that the interactions and the interplay between private sector, nonprofits, and government are opened up, and we use technology, data, social media in order to join forces around problems, then there’s no problem that we face in this country that is not soluble.
Maps and geospatial analysis have become increasingly important as they allow the FCC to display information to the public in an interactive visual format. The FCC’s maps have become useful tools for conveying data in conjunction with Commission reports and public notices. The FCC’s maps site serves as a centralized hub for data visualizations and is one of the most highly trafficked parts of the Commission’s website. Since the launch of the original FCC maps site, a total of 53 maps have been published – including 15 this year – on topics ranging from nationwide LTE coverage to fixed broadband deployment data.
Summary: Today, OMB is releasing an update to Circular A-130, the Federal Government’s governing document for the management of Federal information resources. Today the Office of Management and Budget (OMB) is releasing an update to the Federal Government’s governing document for the management of Federal information resources: Circular A-130, Managing Information as a Strategic Resource. The way we manage information technology (IT), security, data governance, and privacy has rapidly evolved since A-130 was last updated in 2000.
The Census Bureau conducts more surveys than just the Constitutionally-mandated Decennial Census. There is also the American Community Survey, the Economic Census, the County Business Patterns series, statistics on Nonemployer businesses, and the Survey of Business Owners and Self-Employed Persons. On their own, each survey is full of useful information for researchers, local and state governments, and entrepreneurs. However, how valuable would the data be if it were mixed and displayed geographically?
The debate between responsive websites and mobile apps took a decisive turn this week when the United Kingdom’s Digital Service (UKDS) banned the creation of mobile apps. In an interview with GovInsider, the founder of UKDS, Ben Terrett, explained that mobile apps were too expensive to build and maintain. Responsive websites were easier to build and updating the application only requires changing one platform. “For government services that we were providing, the web is a far far better way… and still works on mobile,” Terrett said.
We’re thrilled to announce the Space Apps 2016 Global Award Winners!! These projects well represent the best of the best innovative thinking this year. Congratulations to all the teams. We look forward to seeing you at an upcoming NASA launch in Florida. Best Use of Data: Scintilla, created at the Space Apps Pasadena, California main stage event, mitigates the impact of poor air quality in the global community by democratizing air quality data collection.
Internet strategist Mary Meeker delivered her 2016 Internet Trends report this month, and there are several key takeaways for government agencies to consider and continue tracking as our connected world continues to evolve: Mobile phone adoption and Internet growth is meeting saturation. Incremental global growth will continue (especially in India, which she called out for their wild expansion) but especially for Americans, most people that want to be on the Internet can be on the Internet.
The work of the federal government is incredibly diverse, and affects almost every aspect of American life, whether it is keeping planes in the air or ensuring that our food is safe. Every public service the government provides requires many different skill sets, but the one thing that unites them all is a consistent requirement for transparency. Transparency enables the federal government to demonstrate to the American people how their tax dollars are being used.
Americans Use Public Data to Improve the Lives of Fellow Citizens Data is one of our most important national assets. It informs our policy and our national priorities. But as we have seen time and time again, the most effective way to govern is to engage with the public directly. Thanks to the President’s Executive Order requiring that agencies make data open, we are democratizing access to data.
In its seventh year as home to the U.S. Government’s open data, Data.gov continues to serve millions of people worldwide, from researchers and civic hackers, to businesses and citizens. These users have created apps, launched new products and services, and have improved transparency and openness, making the U.S. Government more accountable and responsive to the American people. Data USA, an online application developed by a team of data scientists at MIT Media Lab and Datawheel, backed by Deloitte is helping Americans visualize demographic and economic data using an open source platform.
My first column when I came back from last year’s summer sabbatical was on the U.S. Patent and Trademark Office’s (USPTO) PatentsView project. PatentsView became one of the Department of Commerce’s most viewed apps in 2015. Building on this success, USPTO released a beta version of its open data portal. The USPTO open data portal is divided into four different sections. The first section leads to patent and trademark datasets.
Last week I spoke at a White House event, “Opportunities & Challenges: Open Police Data and Ensuring the Safety and Security of Victims of Intimate Partner Violence and Sexual Assault.” This event brought together representatives from government agencies, police departments, and advocacy groups to discuss the potential safety and privacy impact of open police data initiatives. The White House launched the Police Data Initiative last year, encouraging police departments to make data sets available to the public in electronic formats that can be downloaded, searched, and analyzed.
The federal workplace is abuzz these days with talk about open data and how agencies can leverage that data through APIs. According to the federal Open Data Policy, data should be managed as an information asset, and making it discoverable and usable (in other words, open). Open data “not only strengthens our democracy and promotes efficiency and effectiveness in government, but also has the potential to create economic opportunity and improve citizens’ quality of life.
It has been over seven years since President Obama signed the executive order that launched the federal open data movement. Much progress has been made, and there is still more to do. Along with the United States, over 100 nations have started programs to provide open access to government data. From large metropolitan governments to small cities, governments are opening up their data to provide better transparency and better delivery of government services.
The NASA Open Innovation team is pleased to announce the availability of the APIs that power Mars Trek and Vesta Trek on api.nasa.gov. The APIs for Mars provide data from the Mars Express, Mars Global Surveyor, Mars Odyssey and Mars Reconnaissance Orbiter missions with 21 different data products such as MOLA Altimetery Hillshade, Viking and THEMIS. There are also 6 data products from the Dawn mission to Vesta providing various views in True Color, Colorized and Color Hillshade to name a few.
The Millennium Challenge Corporation (MCC)’s first open data challenge taught MCC some very valuable lessons in making its public data truly usable by the public. The challenges ask masters and PhD students to find creative ways to use MCC’s publicly-available evaluation data and provide new insights into its evaluation results. As the second challenge launches, MCC is building on these lessons learned from the first challenge: Students are a prime audience for an open data challenge.
Three recent stories demonstrate how opening up federal government data and using agile methods to create federal government software can spur innovation while saving tax money and helping the American public. In its Second Open Government National Action Plan (PDF, 639 KB, 5 pages, September 2014), the White House called for a government-wide policy on open source software. Recently, the Office of Management and Budget released a draft policy “to improve the way custom-developed government code is acquired and distributed moving forward.
Open data and APIs* have not only transformed the federal government; open data and APIs are also transforming tribal, state and local governments. Like federal agencies, some tribal, state and local governments are ahead of other governments in open data innovations. This situation reminds me of my earlier work with the Office of Intergovernmental Affairs in the General Services Administration. In 1998, I was a Presidential Management Fellow working on a project to catalog how state and local governments were using websites to deliver government information and services.
The increasing sophistication of mobile devices has created many opportunities for developers. Thanks to APIs* and open data, developers can build thousands of mobile apps and mobile websites to meet users’ needs. This opportunity has created one of the most contentious debates in the mobile development community: mobile apps versus mobile websites? There is, yet, no solution to the debate. However, there are some advantages and disadvantages to both types of mobile solutions.
This week, President Obama will travel to SxSW (South by Southwest) to talk about how we can use technology to tackle tough challenges. This underscores how important data—government data, in particular—is to improving and fueling our democracy forward. 2015 saw many open data milestones by agencies, including: New advancements in HHS’s syndication storefront New features to analytics.usa.gov dashboard (now with agency-specific dashboards USPTO’s PatentsView Education’s New College Scorecard FEMA’s new Data Visualization Tool APIs from FEC , Labor and NASA (to name a few) There is also more to come (and more that’s needed).
Citizen developers are people who do not work in information technology (IT) but have built IT applications. Back in the mid-80s, business people would smuggle in personal computers to run their spreadsheets and word processing applications (anyone remember VisiCalc and Bank Street Writer?) instead of having to rely on data processing departments. Today, citizen developers use no-code or low-code services such as IFTTT (If This Then That) or QuickBase to build their business apps.
Algorithms are becoming more important as the amount of data grows, and the complexity of government and business processes grows. Put simply, an algorithm is just a set of steps for solving a problem. If you shop online, use an online social network or a mobile app to plan your route, then you are using an algorithm: A sophisticated algorithm that uses large amounts of data to make hundreds (or thousands) of decisions in milliseconds.
For many agencies, what data to make open is left up to the agency’s judgment. This has worked well as agencies do a good job in understanding the public’s needs for specific datasets. Even so, as developers and citizens begin using the open datasets, there is increasing demand for specific agency datasets. The issue is how to best accommodate those requests given the constraints of agency budgets and open data support staff.
The Congressional Research Service recently released a report (PDF, 688 kb, 17 pages, January 2016) describing the big data ecosystem for U.S. agriculture. The purpose of the report was to understand the federal government’s role in emerging big data sources and technologies involved in U.S. agriculture. As the report author, Megan Stubbs, points out, there is not even a standard definition of big data. “Big data may significantly affect many aspects of the agricultural industry although the full extent and nature of its eventual impacts remain uncertain.
Opening up government to better serve the American people has been a key priority of this Administration from day one. On his first full day in office, President Obama signed the Memorandum on Transparency and Open Government, ushering in a new era of open and accountable government. Since then, the Administration has continued to take unprecedented steps to make government more efficient and effective, including launching Data.gov, establishing the international Open Government Partnership, and signing an Executive Order on Making Open and Machine Readable the New Default for Government Information.
With January, and the tearing off of the old calendar, comes the annual taking stock of where we’ve been in the last year and where we can go in the year ahead. So for this month’s editorial theme, we’re taking a closer look at what we think 2016 will bring for digital government—from mobile and content, to open data and accessibility. If our “prognosticators” are correct, this year will be the year when apps become more Web-like; video could overtake social media as the preferred method to communicate; and the number of sensors providing real-time access to (government) data will dramatically increase…just to name a few.
Some highlights from the recent fall conference of the Association for Public Policy Analysis and Management: Localities that receive disaster mitigation funds also have more disaster declarations. Longer, more detailed proposed regulations receive fewer challenges to implementing the regulation. Agencies that are better at quantifying their results are safer from budget cuts. The findings above were all based on ready access to open government data. In fact, many more public policy and administration scholars can do more detailed, and innovative research thanks to federal agencies’ release of these datasets.
2015 was a big year for 18F. We almost doubled in size, worked with 28 different agency partners, and released products ranging from Design Method Cards to cloud.gov. Internally, we improved onboarding and our documentation by releasing guides on topics as diverse as content, accessibility, and creating good open source projects. To mark the end of the year, we reached out to everyone at 18F and asked them to reflect on a meaningful project they worked on this year.
Pop quiz on statistics and data science (answers at the end of the article): 1) I have some data on accidents at railroad crossings. One variable indicates the compass direction a railroad crossing faces (North, Northwest, Northeast, and so on). This variable is a/an: Ordinal Categorical Directional Interval 2) I have some ordinal data that I want to analyze for trends.
A month ago, I wrote about the White House’s call for data scientists and app developers to come together to help combat suicide. On December 12, 2015, there will be five hackathons around the U.S. to #HackSuicide. All the hackathons are free and open to the public. Even if you are not a data scientist, app developer or mental health expert, you may want to attend one of the events to learn how data can be used to solve a vital public health issue.
Some of you may remember when President Reagan opened America’s Global Positioning System (GPS) data. President Reagan gave all countries access to the GPS data in response to the Soviet Union shooting down Korean Airlines Flight 007 on September 1, 1983. I do not believe that the U.S. realized how much opening up GPS data would revolutionize the world economy, health services, travel and almost every other aspect of daily life.
By the time this is published, the United States, along with 160 other countries, will be celebrating Global Entrepreneurship Week (November 16th through November 22nd). November is also National Entrepreneurship Month with November 17th being National Entrepreneurs’ Day. As President Obama stated in his proclamation: “In keeping with our goal of fostering economic growth through private-sector collaboration, the federal government is accelerating the movement of new technologies from the laboratory to the marketplace, increasing access to research awards for small businesses, making more data open to the public [emphasis mine] and catalyzing new industry partnerships in critical fields such as advanced manufacturing and clean energy.
The Data Briefing: White House Asks Data Scientists and App Developers to Help Suicide Prevention Efforts
The White House issued a call on September 30, 2015, for data scientists and app developers to help with a vital public health issue: suicide prevention. From the official announcement: “If you are a data scientist, analyst, tech innovator, or entrepreneur interested in sharing ideas and resources for suicide prevention, we want to hear from you! Please send a brief note about your ideas and resources to mbasco[at]ostp.
Information technology is everywhere. How we communicate, and how we share with one another has gone digital, saving paper, time, money, and making it easier to get information faster and more reliably. Forty-three years ago, when the Clean Water Act was enacted, things moved a little slower. But the significance and impact of this important law remains today. It has helped clean up our lakes and rivers, and ensure that Americans are drinking safe water so we can live active, healthy lives.
OpenNASA has recently completed another redesign of their site. With over 31,000 data sets, 194 code repositories and 36 APIs, OpenNASA probably has the largest collection of open data of any of the federal agencies. An especially helpful feature is a set of icons devoted to five types of visitors: the Citizen Scientist, the Developer, the Citizen Activist, the Govvie and the Curious. A great feature to engage NASA’s audience is the Data Stories section where people talk about the projects they created with NASA datasets.
The Data Briefing: Surrounded by Fields of Federal Data—U.S. Patent and Trademark Office’s PatentsView
Hello, everyone. My summer sabbatical was short but educational, and I am glad to be back in the federal government. I am also excited to again take up the weekly API article that is now expanded to include all things federal government data. Much has happened in the open data realm, and there is much to chronicle as government uses data in more innovative ways. On my sabbatical reading stack was “Code Halos: How the Digital Lives of People, Things, and Organizations are Changing the Rules of Business.
Summary: How the U.S. Digital Service worked with students, families, schools, developers and teams across the federal government to rebuild the new College Scorecard tool. My niece is a smart kid. I’m biased, but I swear she is. And just as I started working on the College Scorecard project as the U.S. Digital Service’s new Chief Digital Service Officer at the Department of Education, I got a call from her—she was trying to decide where to go to school.
A few weeks ago the Council’s Innovation Committee released the Open Data Prioritization Toolkit. The response to the toolkit has been positive, but we also heard back from the community asking why the Open Data toolkit’s summary was locked up in a non-open format—PDF. The Council Operations Team noted the irony of publishing a guide to opening data in a non-open format made a decision to eat our own dogfood.
Challenge.gov offers a number of services to help agencies create successful competitions. One challenge that recently wrapped up made use of the full range of these services to come up with some creative, useful apps that have nationwide implications. Presidential Innovation Fellow Jeff Meisel led the CitySDK (Software Development Kit) launch. The team wanted a different way to reach data consumers. The U.S. Census Bureau wanted to find a new way to create the most innovative data-driven apps sparking change in cities from coast to coast.
The National Institute of Standards and Technology (NIST) wants YOU to help them build native apps. NIST launched the Reference Data Challenge to improve the way the agency shares scientific reference data. They want third party developers from around the country to build native apps that aggregate and improve the usability of free NIST datasets and resources. They are offering $45,000 in prize money and are taking submissions until the end of September.
Behind every great innovation is a team. And behind successful innovation teams are efficient tools, processes, and most importantly, people. The Department of Energy’s SunShot Initiative funds projects that make solar energy more affordable and accessible for Americans. As part of the initiative, the SunShot Catalyst open innovation program seeks to rapidly deliver solar solutions through prize challenges. Catalyst has been recognized as a leader in the innovation field. The International Society for Professional Innovation Management (ISPIM) recently awarded Catalyst the ISPIM Grand Prize 2015 for excellence in innovation management.
Digital communities of practice come in many stripes. DigitalGov communities span eight (and counting) focus areas and have thousands of members, but strong collaborations exist in all corners of government. In honor of this month’s communities theme, we are offering a list of communities that foster connections and strengthen the digital capabilities of federal agencies. Here is a list of some communities working in the digital arena: 18F /Developer Program CIO Council: Accessibility Community of Practice CIO Council: Privacy Community of Practice Crowdsourcing and Citizen Science Drupal for Government eCPIC Federal Steering Committee (FESCom) Federal Communicators Network Federal Intranet Content Managers Federal Knowledge Management Community Federal Librarians Ideation Community of Practice Mobile Health (m Health) Training Institutes Training Institutes”) Open Data listserv: Anyone with a .
You have the right to a safe workplace—and so do the employees at your favorite café, the local hospital and the construction company renovating homes in your neighborhood. But how can you tell if the businesses you patronize are keeping their workers safe? That’s a question we can answer with data. The Data The Occupational Safety and Health Administration’s online enforcement database includes details on the roughly 90,000 OSHA inspections conducted every year, and covers more than four decades.
Take out your smartphone and count the number of apps that you have. How many of these apps do you use daily? What about the apps you use weekly? Do you have any apps that you installed but used only once? Any apps that you have never used? What kind of apps do you have? Are most of the apps used to communicate with friends and family? How many gaming apps do you have?
Civic hackers are a special breed—their primary motivation is closely tied to the social issues closest to their hearts. Most attend hack-a-thons, engage in civic meetups, and show up at city hearings to champion their cause and push solutions at the societal, technology, and policy levels. On the technological front, creating civic city-based solutions has traditionally been unnecessarily difficult. Data issues range from the lack of open data access to the inconsistent interpretation of current data sets to the difficulty of using federal data, such as U.
Over the last several years, continuing advances in computer processing power and storage have brought about the growth of what some call big data. Mobile and wearable devices now also generate large amounts of data via our interaction with various apps and our geographic location. This endless stream of information is being harnessed to create extremely informative dashboards like analytics.usa.gov and helping make advances in medicine and even farming possible.
The National Day of Civic Hacking was born when some of the nation’s leaders in civic engagement decided to rally around a common goal on one weekend. -Nicholas Skytland, NASA The National Day of Civic Hacking is a national community engagement event that will take place on June 6, 2015, in cities all around America. The initiative is a united effort to bring together a diverse group of concerned citizens to improve communities and the government which represents them.
The Pew Research Center just released a report on how Americans view open government data. The following findings were based on a November to December 2014 survey of 3,212 adults. Two-thirds of Americans use the Internet or an app to connect with the government. According to Pew, 37% use the Internet to connect with the federal government, 34% connect with their state government, and 32% connect with their local government.
I grew up when home computers were first being introduced to the general public. I bought my first computer, a Commodore 64, after spending a summer of mowing lawns and saving up my birthday and Christmas money. It was not until I entered college that I became an infopreneur. Infopreneurs are entrepreneurs who used computers and data sources to provide information products and services. My specialty was compiling information from the university’s collection of CD-ROMs that they received from various government agencies.
In case you missed it, U.S. Open Data recently launched a tool called: Let Me Get That Data For You (LMGTDY). The name is a play on the very funny Let Me Google That For You website. How LMGTDFY works Let Me Get That Data For You searches any website for data in machine-readable formats and provides a list. Here is U.S. Open Data’s background reasoning for creating this tool:
The API Briefing: Free Federal Energy and Economic Information Delivered Straight to Your Spreadsheet
Back in November 2014, I wrote about the Federal Reserve of St. Louis’ FRED® (Federal Reserve Economic Data) API. A user can access 238,000 economic trends through FRED® through a website and mobile apps. What is unique about FRED® is that a user can pull economic data directly into an Excel spreadsheet. Now, the FRED® Excel plugin is joined by the U.S. Energy Information Administration’s (EIA) Excel plugin. The tool, which launched on March 18, incorporates both energy data from the EIA API and economic data from FRED®.
Metadata for website content is usually managed as part of the editorial process when documents are created and published with content management systems. There may be another source for this metadata, especially in regulatory agencies: internal databases that reference Web content in support of record keeping processes. These databases may contain public and non-public information that were never meant to be published for public consumption. “Metadata” is not typically how the content is described.
At the U.S. Agency for International Development (USAID), our new open data policy will begin making more Agency-funded data broadly accessible to the public. It completely changes the way we do business, and it also means that in the coming years, the amount of data we host on our open data website (known as the Development Data Library) will dramatically increase. So the question is: when we’re done overhauling our website, how will the user make sense of all that information to find exactly what they’re looking for?
Many forces are converging to strengthen the political, economic and commercial ties that bind the United States, Canada and Mexico. The GSA Office of Citizen Services and Innovative Technologies (OCSIT) has anticipated this drive toward collaboration for decades, building a network of links among the three nations’ Chief Information Officers and other national technology and data experts. Annual OCSIT-sponsored North American Day (NAD) talks have contributed to improved digital services in all three countries.
DigitalGov University has hosted some great events over the last year in partnership with Data.gov, the MobileGov Community and 18F to bring you information on opening data and building APIs. This month we’ve rounded up the events over the past year so that you can see what’s been offered. Use the comments below to offer up suggestions on what else you’d like to see on the schedule.
Data and code are the foundation, building blocks, and cornerstone of government digital services. They are the keys that open the door to a better digital government future and are fundamental in making government more open. No matter who you are or where you work in the federal space, data and code enable your projects to meet real needs. This month we’re featuring articles around the theme of data and code.
The federal government has IT challenges, and innovative federal projects are tackling those challenges head-on. As projects gain momentum, outside organizations have taken notice. Recently, Data.gov and DigitalGov’s Digital Analytics Program (DAP) were recognized by the American Council for Technology and Industry Advisory Council (ACT-IAC), among 30 other finalists for the Igniting Innovation Award. ACT-IAC’s 2015 Igniting Innovation Showcase and Awards recognized tools, solutions, services and programs developed by government and industry leaders.
Tackling technology tasks just got easier. Recently, federal agencies negotiated eight new Terms of Service (TOS) Agreements for free apps and services. DigitalGov has an extensive list of federal-friendly TOS agreements for free products, and the list is updated as new TOS agreements are created. Cyfe Cyfe, a business dashboard app, helps users monitor diverse data streams in one location. It displays information related to social media, analytics, marketing, sales, support, and infrastructure tools.
Smartphone adoption rate continues to rise, but the screen sizes users adopt continue to evolve. According to preliminary data from the International Data Corporation (IDC) Worldwide Quarterly Mobile Phone Tracker, smartphone vendors shipped a total of 375.2 million units during the fourth quarter of 2014. IDC states that this was an increase of more than 25%, compared to the fourth quarter in 2013. For the full year, IDC says the worldwide smartphone market saw a total of 1.
Conserving energy is not a shot in the dark. Millions of people can now shine a light on their electricity usage as a result of a dynamic public-private partnership based on open data. The goal of the Green Button Initiative is to provide electricity customers with access to their energy usage data in an easy-to-understand and computer-friendly format. Customers can click on the “Green Button” logo on participating companies’ websites and download their personal energy use information.
For months, I’ve been trying to figure out how to get leads for the OMB External Use Open Data Survey responses. I’ve attended Google Analytics seminars, asked for survey responses from some of our public facing sites, added a data request form to our /data page, and begged for leads from program owners. The result was very few leads and no indication of whether or not they were people who actually were looking for our data, used our data or just had a website resource access issue.
Fighting malaria in Botswana with a group of high school students in D.C. Contributing to the Ebola response from the West Bank. These scenarios may not fit the typical image of humanitarian aid efforts, but technology has transformed the possibilities for public participation in international development. Crowdsourced mapping projects have become key contributors in relief efforts and highlight the collaborative work that can be done between government, non-profit organizations, and the public.
Criminal justice agencies collect a variety of information and use it in multiple ways. Having a clear understanding of current realities is critical to shaping policies and improving the administration of justice. Police use data to identify hot spots; judges use it when they impose sentences; victim assistance staff use it to provide better services. The problem is that criminal justice datasets are often large, complex, and contain a wide variety of geocodes and identifiers.
Data.gov is the central clearinghouse for open data from the United States federal government. It also provides access to many local government and non-federal open data resources. But how does this data get on to Data.gov? Data.gov does not host data directly, but rather aggregates metadata about open data resources in one centralized location. In addition to Data.gov’s recent webinars on how Data.gov harvests data, the Data.gov team has created a living resource to explain in further detail how data must be structured to connect to the Data.
In case you missed it: the Data.gov team recently hosted DigitalGov University webinars designed to help agencies and open data advocates better understand how to get data on Data.gov and how to implement the Open Data Policy’s metadata schema updates. These webinars were designed assist government data publishers in making more data discoverable to the American people. You can watch these webinars and check out additional supplemental resources below.
Thanks to the power of open data and APIs, federal agencies can now register their mobile native apps and websites on the Federal Mobile Products Registry and have them appear on the USA.gov Federal Mobile Apps Directory (formerly USA.gov Apps Gallery) almost immediately. When we launched the USA.gov Apps Gallery in 2010 there were less than 15 apps. To register an app, an agency would contact us with app info, download screenshots and create a “page” for that app.
We are all collecting a lot of performance data across our digital properties and DigitalGov University has hosted many events on the collection, reporting and strategizing around metrics. DigitalGov has shared many posts on these topics as well. So we thought it would be great to curate these events and posts for easy reference and sharing. Data Collection If you are having a hard time measuring the success of your social media efforts, email campaigns or even your website, these are some of the events that’ll help.
The National Archives and Records Administration and Wikimedia D.C., invite you to help us improve access to open government data on Wikipedia. We are excited to announce that we will be hosting the Open Government #WikiHack, a two-day hackathon at the National Archives Building in downtown D.C., over the weekend of September 27 and 28. Did you know that Wikipedia articles with NARA digital images saw over 1 billion page views in FY13 alone?
As traffic to desktop .gov websites declines, how we publish our content increasingly matters. We need to meet people where they are as they seek information on the Internet. To do so, we need to adjust to the new world of mobile applications, social media, and instant answers provided by search engines. Freeing Content from Our Websites In this content sharing era, it is important to separate the content from how it appears on your site.
Campaign finance information is not very approachable, even when made available as open data. The laws that regulate how money can be spent around elections are important to our democracy, but sometimes it’s difficult to understand how these laws apply. Between Senate, House, and Presidential campaigns, thousands of people run for office on a regular basis (every two years for the House of Representatives, every six years for the Senate, and every four years for the Presidency).
How can you find the top 5 users of your open data? We were recently asked this question on the Open Data listserv, and while this information can be a good measure of success for open data programs, we also figured some of the answers shared would be of interest to the broader community. This blog post seeks to summarize and clarify those answers. What Defines a Top Third-Party Developer?
More than 100 digital engagement and open data managers from across government met with leaders in the private sector startup community August 7 at the White House for a summit on integrating our digital services with public participation to create more opportunities for innovation and tackle tougher challenges. The SocialGov Summit on Open Data Innovation was organized by the 700-member SocialGov Community and White House Office of Science and Technology Policy, launching a new inter-community initiative to apply combined open data, digital engagement, and innovative technologies to fields ranging from the Internet of Things and emergency management to modernization of the regulatory process.
The U.S. Census Bureau today released Census PoP Quiz, a new interactive mobile application that challenges users’ knowledge of demographic facts for all 50 states and the District of Columbia. The new app, which draws from the Census Bureau’s American Community Survey, aims to raise statistical literacy about the U.S. population. Census PoP Quiz provides an introduction to the statistics that define our growing, changing nation and is a great way for everyone to learn facts about all 50 states, the District of Columbia and the nation in a fun, relevant way.
Open data and big data—and the responsible management and protection of that data—are key components of the President’s agenda to drive innovation and economic growth. On Thursday, June 19, leaders from civil society, industry, academia, and 40 federal departments and agencies met at Georgetown University’s McCourt School of Public Policy’s Massive Data Institute to discuss how federal agencies can continue to unlock government data to drive innovation and improve services. Drawing from the White House Working Group report, Big Data: Seizing Opportunities, Preserving Values, this event focused on opening and using government data, while appropriately protecting privacy and preventing the use of data to discriminate against vulnerable populations in our society.
This morning I was walking down 18th Street, crossing Pennsylvania Avenue by the World Bank when I heard what sounded like “a test from the Emergency Broadcast System.” I looked behind me and realized it was coming from my purse and that my phone was jiggling. I pulled out my phone to see that there was a flash flood warning. I looked up and saw dozens of people on the crowded sidewalks pulling out devices.
Last June, President Obama launched a Climate Action Plan to cut carbon pollution, prepare communities for the impacts of climate change, and lead international efforts to address this global challenge. The plan recognizes that even as we act to curb the carbon pollution that is driving climate change, we must also prepare our citizens and communities for the climate impacts that are already underway across the country. One of the efforts described in that Climate Action Plan is the Climate Data Initiative, a broad effort to leverage the federal government’s extensive, freely-available climate-relevant data resources data to spur innovation and private-sector entrepreneurship in order to advance awareness of and preparedness for the impacts of climate change.
In May 2009, Data.gov was an experiment. There were questions: would people use the data? would agencies share the data? and would it make a difference? We’ve all come a long, long way to answering those questions, starting with only 47 datasets and having 105,000 datasets today. We realized that this was never simply about opening up government data, but rather about growing and nurturing an open data ecosystem to improve the lives of citizens.
One year ago today, President Obama signed an executive order that made open and machine-readable data the new default for government information. This historic step is helping to make government-held data more accessible to the public and to entrepreneurs while appropriately safeguarding sensitive information and rigorously protecting privacy. Freely available data from the U.S. government is an important national resource, serving as fuel for entrepreneurship, innovation, scientific discovery, and economic growth.
As the definition of “developer” has grown and expanded, GitHub has become a place where anyone can do simple collaboration. It’s a free social network that tracks changes to any data, not just code, where stakeholders and developers can work on the same data simultaneously. Project Open Data, a cross-agency initiative developed by the White House, that looks at how to manage information as an asset in the 21st century, is powered by GitHub.
We’ve written a few times about the changes that we’ve been working on for Data.gov to make it easier for users to find, understand, and use government data. Today you’ll notice even more changes to Data.gov – here’s a quick rundown of some of the main changes you’ll see, and why. Works on your mobile device The site is now responsive to the device you’re using. Pull up Data.
Since the launch of Next.Data.gov, your help and ideas have made it possible to make two updates to the site. We’re naming these biweekly releases after the presidents so the one that launched this week is the Adams Release. We’re pleased to announce that much of the work was done by the Data.gov Presidential Innovation Fellow, Dave Caraway, whose passion is open data and how it can be used by entrepreneurs to build businesses and create jobs.
Americans are rocking open data! From getting people to the emergency room faster with iTriage to helping them navigate road and rail after a disaster, people are innovating, building businesses, and creating safer communities. As developers get more sophisticated and businesses get better analytics, Data.gov needs to change to support them in new ways and your ideas will help to build that future. You are invited to create that new vision.
As you know, last month Data.gov launched its new open-source Data.gov 2.0 catalog (catalog.data.gov). Based on CKAN, a data management platform used by many open-data catalogs around the world, Data.gov’s new catalog has received nothing but kudos from users. For the first time, our raw datasets, tools and geospatial datasets are in one place making search and discovery easier than ever. To make exploring the new catalog even easier, Data.
Data.gov launched a major upgrade today, moving to a new catalog based on an open source data management system calledCKAN. In the process of migrating to a new data catalog, Data.gov had the opportunity to do another round of usability testing. Lucky for us, the DigitalGov User Experience Program, that teaches agencies how to test federal websites, is right in our own backyard. With today’s launch, you’ll see the initial results of what we learned from our testers; an expert Data.
Last week, we told you about the upcoming relaunch of Healthcare.gov and its use of the Jekyll website generator. Jekyll allows users to build dynamic websites served by static pages. To help manage large websites using Jekyll, developers working on the new healthcare.gov published the ‘Prose.io’ editing interface last year. Content editors will use this lightweight editor to create and manage content across the site. Prose is an open source web application that allows users to manage web content stored on GitHub’s code sharing service.
What Are APIs? An Application Programming Interface, or API, is a set of software instructions and standards that allows machine to machine communication—like when a website uses a widget to share a link on Twitter or Facebook. [youtube http://www.youtube.com/watch?v=BVeiRCEwJx8] [Extended Version] When we talk about APIs we are referring to Web services or Web APIs. This aligns with the current trends to use Web APIs to support sharing content and data between communities and applications.
One way agencies can offer APIs for their data is to use the built–in functionality of Data.gov. The information that is hosted as interactive datasets have an API layer which agencies can make available through documentation in the developer’s section of the agency’s website. The guide below will help you do this. Process Upload a dataset as an interactive dataset in Data.gov. A. Your agency should have a Data.
After choosing a set of information or services to offer via API, some of your next steps are to plan and implement the API. You’ll still need to prepare documentation, tools, and other elements that make a complete package for the API, but at the center is the actual Web service itself. In many situations, existing IT resources or the current system operators handle this step. There are several options at this stage, each with advantages and disadvantages.