It’s been a while since I’ve checked in on enterprise architecture (EA). My last in-depth work with EA was around 2011 when I was on detail to the Office of Personnel Management’s Open Government Team. The EA model I worked with was the top-down organizational design of information technology assets, data assets, and business processes. Many of you are probably familiar with this traditional EA model. Six years later, it is predicted that in 2018 that “half of enterprise architecture (EA) business architecture initiatives will focus on defining and enabling digital business platform strategies.
Last week, the Federal Election Commission (FEC) unveiled their new website at FEC.gov. This new site is the result of a years-long collaboration with GSA’s 18F and features completely revamped tools for exploring campaign finance data. It provides user-centered content for understanding the reporting and compliance requirements for people participating in federal elections, redesigned tools for exploring legal resources, and more. Why it matters On the agency’s “About the FEC” page, it says, “The FEC was created to promote confidence and participation in the democratic process.
The Department of Education (ED) launched its first developer site. The developer site is built on GitHub which will make it easier for ED to centralize their code and Application Programming Interfaces (APIs). Currently, ten APIs are on the developer site: The Civil Right Data Collection (CRDC) APIs: These three APIs give information on public school enrollment in 2013–14, chronic absenteeism in 2013–14, and out-of-school suspension in 2013–14. The College Scorecard API: This is data from the College Scorecard project which allows student and families to “compare college costs and outcomes as they weigh the tradeoffs of different colleges, accounting for their own needs and educational goals.
Last [month], NASA Open Innovation Program Manager Dr. Beth Beck and her team traveled to the Canadian Space Agency (CSA) near Montreal, Canada to attend the Inspiring Data Forum graciously hosted by our Open Data neighbors to the North. The goal of this gathering was to bolster the working relationship between the two Space Agency’s Open Data efforts and to present techniques NASA is doing in Open Innovation. The event was heavily attended by CSA employees and also had participants from National Research Council of Canada, Treasury Board Secretariat, Natural Resources Canada, Agricultural and Agri-Food Canada, MaxQ and Social Sciences & Humanities Research Council of Canada.
DigitalGov University (DGU), the events platform for DigitalGov, provides programming to build and accelerate digital capacity by providing webinars and in-person events highlighting innovations, case studies, tools, and resources. Thanks to your participation, DGU hosted over 90 events with 6,648 attendees from over 100 agencies across federal, tribal, state, and local governments. DGU strives to provide training throughout the year that is useful and relevant to you. One of the most resounding comments from digital managers last year was people wanted to be able to attend all of our classes virtually.
When: Friday, December 9th, 2016 Where: NARA Innovation Hub, 700 Pennsylvania Avenue, NW Register: On Eventbrite The U.S. General Services Administration (GSA) Digital Service team is excited to be hosting our next agency hackathon on December 9, 2016. Join coders from across the region as we come together in celebration of Grace Hopper’s birthday. Admiral “Amazing Grace” Hopper was one of the first programmers in the history of computers. As the creator of the first compiler for a programming language, it is largely due to her that programmers use “if/thens” instead of 1s and 0s today.
Cancer clinical trials are a critically important step on the pathway for new or improved treatments to make their way to patients in clinics and hospitals in towns and cities across the country. Patients and their loved ones are relying on these rigorous studies to determine whether promising new therapies and approaches might extend how long they live or improve their quality of life. For many years, a steady number of patients with cancer, approximately 5%, have participated in cancer clinical trials.
Daily imagery data taken by the Earth Polychromatic Imaging Camera is now accessible via a RESTful API available from the NASA API Portal. The Earth Polychromatic Imaging Camera (EPIC) is an instrument aboard NOAA’s DSCOVR satellite, which orbits at Earth’s Lagrange point, the sweet spot in space where the gravitational tug of the Earth and the Sun is equal. This allows DSCOVR to maintain a stable position between the Earth and Sun and thus a continuous view of the sunlit side of Earth.
Here is the outline for our 2016 Open Government Plan. Let us know what you think. We’ve also posted this on GitHub/NASA for your comments: https://github.com/nasa/Open-Gov-Plan-v4. NASA and Open Government NASA is an open government agency based on the founding legislation in the National Aeronautics and Space Act of 1958, which calls for participation and sharing in the conduct of how we go about the business of expanding the frontiers of knowledge, advancing understanding of the universe, and serving the American public.
The Data Briefing: Microservices and Serverless Apps — A New Direction for Federal Government Mobile Apps?
Continuing from last week’s column on DevOps and containers, I will explain two other hot trends in IT — microservices and serverless apps. For those who want official federal government guidance, the National Institute of Standards and Technology (NIST) has released a draft special publication on microservices, application containers, and system virtual machines (PDF, 660 kb, 12 pages, February 2016). I wrote about microservices and containers in February 2015 as two API* trends to watch.
The Census Bureau conducts more surveys than just the Constitutionally-mandated Decennial Census. There is also the American Community Survey, the Economic Census, the County Business Patterns series, statistics on Nonemployer businesses, and the Survey of Business Owners and Self-Employed Persons. On their own, each survey is full of useful information for researchers, local and state governments, and entrepreneurs. However, how valuable would the data be if it were mixed and displayed geographically?
It is at the intersections of fields where you find the most fascinating and innovative concepts. Recently, a conference on “Open Human Resources and the Cognitive Era” explored the use of chatbots and blockchain technologies in human resources. Human Resources (HR) is quietly undergoing a revolution as many HR practitioners are transforming HR by using open source concepts. It is fascinating to see how cognitive technologies and cloud technologies are changing HR from a transactional and compliance function to an essential strategic organizational asset.
****Gray Brooks of GSA gave us a great definition of APIs in the DigitalGov University (DGU) presentation, Introduction to APIs. He described APIs as “Application Programming Interfaces, or APIs, are web services that allow people to more easily consume content and data in multiple ways—via mobile devices, mobile apps, innovative mash-ups, and much more.” Simply put, “APIs are a better way to get government information and services into the hands of the people who need them.
The National Archives is pleased to participate in the U.S. Digital Registry, the authoritative resource for official third-party websites, social media platforms and mobile apps managed by the U.S. federal government. The U.S. Digital Registry is an API-generating platform designed to authenticate third-party sites in the federal government in order to help maintain accountability over our digital services. As more users access services, communicate, and engage with their government online and through social media, the U.
Americans Use Public Data to Improve the Lives of Fellow Citizens Data is one of our most important national assets. It informs our policy and our national priorities. But as we have seen time and time again, the most effective way to govern is to engage with the public directly. Thanks to the President’s Executive Order requiring that agencies make data open, we are democratizing access to data.
My first column when I came back from last year’s summer sabbatical was on the U.S. Patent and Trademark Office’s (USPTO) PatentsView project. PatentsView became one of the Department of Commerce’s most viewed apps in 2015. Building on this success, USPTO released a beta version of its open data portal. The USPTO open data portal is divided into four different sections. The first section leads to patent and trademark datasets.
The federal workplace is abuzz these days with talk about open data and how agencies can leverage that data through APIs. According to the federal Open Data Policy, data should be managed as an information asset, and making it discoverable and usable (in other words, open). Open data “not only strengthens our democracy and promotes efficiency and effectiveness in government, but also has the potential to create economic opportunity and improve citizens’ quality of life.
Ten months ago, I wrote about the rise of the post-app world in which mobile personal assistants would do the work of five to 10 apps combined. These mobile personal assistants, now known as chatbots, would work through conversational interfaces (voice and instant messaging, for example). The idea is to build more natural interfaces for people to access information services and perform complicated online tasks. Facebook has now joined in the new conversational commerce marketspace along with Google and Apple.
The NASA Open Innovation team is pleased to announce the availability of the APIs that power Mars Trek and Vesta Trek on api.nasa.gov. The APIs for Mars provide data from the Mars Express, Mars Global Surveyor, Mars Odyssey and Mars Reconnaissance Orbiter missions with 21 different data products such as MOLA Altimetery Hillshade, Viking and THEMIS. There are also 6 data products from the Dawn mission to Vesta providing various views in True Color, Colorized and Color Hillshade to name a few.
Three recent stories demonstrate how opening up federal government data and using agile methods to create federal government software can spur innovation while saving tax money and helping the American public. In its Second Open Government National Action Plan (PDF, 639 KB, 5 pages, September 2014), the White House called for a government-wide policy on open source software. Recently, the Office of Management and Budget released a draft policy “to improve the way custom-developed government code is acquired and distributed moving forward.
Open data and APIs* have not only transformed the federal government; open data and APIs are also transforming tribal, state and local governments. Like federal agencies, some tribal, state and local governments are ahead of other governments in open data innovations. This situation reminds me of my earlier work with the Office of Intergovernmental Affairs in the General Services Administration. In 1998, I was a Presidential Management Fellow working on a project to catalog how state and local governments were using websites to deliver government information and services.
The increasing sophistication of mobile devices has created many opportunities for developers. Thanks to APIs* and open data, developers can build thousands of mobile apps and mobile websites to meet users’ needs. This opportunity has created one of the most contentious debates in the mobile development community: mobile apps versus mobile websites? There is, yet, no solution to the debate. However, there are some advantages and disadvantages to both types of mobile solutions.
Citizen developers are people who do not work in information technology (IT) but have built IT applications. Back in the mid-80s, business people would smuggle in personal computers to run their spreadsheets and word processing applications (anyone remember VisiCalc and Bank Street Writer?) instead of having to rely on data processing departments. Today, citizen developers use no-code or low-code services such as IFTTT (If This Then That) or QuickBase to build their business apps.
Here at DigitalGov, we generally focus on federal governmental digital efforts within the U.S. It is where we live and operate, so it makes sense, but many governments across the world struggle with the same issues and leverage technology as a common solution. When I came across an article where Australia announced its “government as an API” platform was available, it seemed like a great opportunity to see how another country is tackling structured and open content.
The Federal Election Commission (FEC) empowers citizens with the information they need to make informed decisions about their democracy. Since opening its doors in the ‘70s, the FEC has evolved to better serve the public with that information. As the years progressed, records have gone from paper to microfilm and eventually to the web. Today marks the launch of the FEC’s first API. With that API, searching for candidates and committees will be easier and more interactive.
You have the right to a safe workplace—and so do the employees at your favorite café, the local hospital and the construction company renovating homes in your neighborhood. But how can you tell if the businesses you patronize are keeping their workers safe? That’s a question we can answer with data. The Data The Occupational Safety and Health Administration’s online enforcement database includes details on the roughly 90,000 OSHA inspections conducted every year, and covers more than four decades.
Take out your smartphone and count the number of apps that you have. How many of these apps do you use daily? What about the apps you use weekly? Do you have any apps that you installed but used only once? Any apps that you have never used? What kind of apps do you have? Are most of the apps used to communicate with friends and family? How many gaming apps do you have?
Civic hackers are a special breed—their primary motivation is closely tied to the social issues closest to their hearts. Most attend hack-a-thons, engage in civic meetups, and show up at city hearings to champion their cause and push solutions at the societal, technology, and policy levels. On the technological front, creating civic city-based solutions has traditionally been unnecessarily difficult. Data issues range from the lack of open data access to the inconsistent interpretation of current data sets to the difficulty of using federal data, such as U.
By now, you are familiar with “big data” or datasets that are so large that they cannot be analyzed by conventional analytical methods. You may have heard of “long data” which is data that has a temporal context. I work with long data when I analyze hiring patterns over time in workforce data. There is also “small data.” Small data are datasets that describe a current condition. For example, if you have a smart home appliance such as a smart thermostat or a home security system, that appliance is constantly monitoring data such as temperature or if a door is open.
Serendipity can be a wonderful tool for discovery. I was looking through the Census Bureau site for some business census data when I came upon the 2012 Census of Governments. According to the official description: “[t]he Census of Governments identifies the scope and nature of the nation’s state and local government sector; provides authoritative benchmark figures of public finance and public employment; classifies local government organizations, powers, and activities; and measures federal, state, and local fiscal relationships.
NASA has been busy since we last visited their collection of APIs back in August 2014. NASA has just launched API.NASA.gov where developers can learn to use existing NASA APIs or contribute their APIs to the catalog. NASA encourages developers to obtain an API Key to begin using or contributing APIs. Developers do not need an API key, but their requests to the API will be limited. I would encourage developers to obtain an API key.
The spring semesters are winding down at the universities where I teach. Many students are looking for summer internships or their first job after graduation. Of course, I talk about the opportunities in government through the Pathways program, the Presidential Management Fellows, or the various agency-specific internship programs. I’ve demonstrated USAJOBS in my classes, but I often wondered how to improve the experience for job seekers, especially for job seekers who prefer to use mobile apps.
Before coming to DC in late 2008, I lived in Louisville, Kentucky. Louisville is in the Ohio Valley Region, which meteorologists euphemistically call “weather-rich.” With spring came the beautiful flowers and the Kentucky Derby. Spring also brought flooding, tornadoes, severe thunderstorms, and windstorms. This is why I had several emergency weather radios that also doubled as flashlights and cell phone chargers. I also have several emergency information apps on my smartphone.
The Pew Research Center just released a report on how Americans view open government data. The following findings were based on a November to December 2014 survey of 3,212 adults. Two-thirds of Americans use the Internet or an app to connect with the government. According to Pew, 37% use the Internet to connect with the federal government, 34% connect with their state government, and 32% connect with their local government.
I recently found an app that provides a great service through crowdsourcing. Be My Eyes connects visually-impaired people with volunteers. Using the smartphone’s camera, the volunteers can perform tasks such as reading an expiration date or helping someone navigate unfamiliar surroundings. This is not a federal app, but I wanted to highlight it to demonstrate how crowdsourcing apps can make it easy for everyone to make a difference through microtasks.
Four years ago, we released our Labor Department-wide API—that is, an Application Programming Interface—with the hope that anyone who wants to build an app using our data could do so easily. At the time, we started off with three datasets. Today, we have around 200, including workplace injuries and illnesses, the unemployment rate, companies’ compliance with wage and hour laws, and many other important topics. We hope that the data we publish can help people who have jobs, are looking for jobs or are even retired from their jobs.
Data is one of the most important assets at NASA. We have data on comets, measurements of Mars, and real-time imagery of Earth. But what good is data if you can’t access it? Not good at all! We’re in the process of building a site (at api.nasa.gov) to catalog NASA APIs that structure access to our data, making it eminently easy for developers to build applications. An application, here, is broadly defined and includes research applications, mobile applications, policy applications—any data use that converts information into insight and action.
I grew up when home computers were first being introduced to the general public. I bought my first computer, a Commodore 64, after spending a summer of mowing lawns and saving up my birthday and Christmas money. It was not until I entered college that I became an infopreneur. Infopreneurs are entrepreneurs who used computers and data sources to provide information products and services. My specialty was compiling information from the university’s collection of CD-ROMs that they received from various government agencies.
When I first started coding using BASIC on the Commodore 64, I rarely documented my programs. Neither did many of my fellow programmers which led to numerous hours trying to figure out just exactly how a program worked. Documentation became more vital as programs became more feature rich and complex. In the API world, there are a number of documentation standards to choose from when documenting an API.
Ever since we announced IFTTT was available for federal use, dozens of ideas have been shared for how program managers can use the tool to increase their productivity. I asked some API enthusiasts in the SocialGov community which of their favorite recipes were must-haves for all digital teams or for those new to the platform. First, for those not familiar with it, IFTTT (as in “If This Then That”) combines 166 channels like Twitter, Android and iOS Location, and RSS into “recipes” that can integrate government social media, data, location-based services, and the Internet of Things.
Finding and getting access to our own health information can be a complex process. And most of us don’t really think about having our health information readily accessible until we really need it – like in the event of an emergency, or when switching doctors or traveling. Combing through stacks of paperwork and contacting providers is daunting for even the most organized among us. However, this familiar scenario is being reimagined.
The API Briefing: Free Federal Energy and Economic Information Delivered Straight to Your Spreadsheet
Back in November 2014, I wrote about the Federal Reserve of St. Louis’ FRED® (Federal Reserve Economic Data) API. A user can access 238,000 economic trends through FRED® through a website and mobile apps. What is unique about FRED® is that a user can pull economic data directly into an Excel spreadsheet. Now, the FRED® Excel plugin is joined by the U.S. Energy Information Administration’s (EIA) Excel plugin. The tool, which launched on March 18, incorporates both energy data from the EIA API and economic data from FRED®.
Instead of writing about a specific federal API this week, I want to talk about a new, evolving way of building Web interfaces and complete applications. Web Components allow developers to create their element that extends the HTML5 set of tags. Developers can create a Web Component that is a button that performs a specific function, such as composing and sending an email. Alternatively, a Web Component can be a complete application that a developer can easily drop into a Web page or mobile app.
Anything built should be built right. It doesn’t matter if it’s built of wood, carbon nanotubes or code. So it’s encouraging that the practice of User-Centered Design—getting customer feedback at every stage of a project—is catching on with APIs as well. When we think APIs, we mostly think of developers and not designers. But the experience of those who want to use your APIs isn’t just dependant of the strength and elegance of your API.
The API Briefing: Fulfilling the D(e)SIRE for Renewable Energy with the Department of Energy’s New API
The Database of State Incentives for Renewables and Efficiency® (DSIRE®) provides information on incentives and policies for renewables and energy efficiency in the U.S. This joint project by the Department of Energy and North Carolina State University just released an API to query DSIRE®’s database. Developers can view the sample output by visiting the database query page. They can query by state or ZIP code to receive a listing of programs.
When browsing the various APIs offered by the federal government, you may have noticed that developers need to sign up for an API key. You may have also noticed that the documentation tells app developers to access the API using specified methods. Along with these two requirements, federal API creators have several ways to provide secure APIs for app developers and the general public. In this posting, I will describe how federal APIs are kept secure.
APIs and apps have been created for almost every aspect of human life. There are alarm clock apps, fitness apps, cooking apps, and personal finance apps, just to name a few of the thousands of apps available today. Most areas of society are well-represented in the app world except for one large portion of the American public—rural America. There need to be more apps for rural America. Fortunately, the U.
In my last posting, I argued that federal agencies should consider microservices architecture when releasing APIs. This is because allowing users to combine single-purpose apps together in unique ways helps people build personalized apps such as a driving map to local farmers markets. When given the opportunity, users will surprise you with the innovative creations they build from combining APIs. Just last week, the popular If This Then That (IFTTT) service released a federal-friendly Terms of Service.
DigitalGov recently spotlighted the National Highway Traffic Safety Administration’s (NHTSA) new SaferRide app. SaferRide provides safe alternatives to keep drunken drivers off the road. SaferRide uses the Yelp API to provide information about local taxi services for the part of the app where users can request a ride home. The SaferRide app is one example of how APIs can be mashed together to produce sophisticated applications. As APIs become more prevalent, there are two trends in app development that federal app developers should watch.
Big news in the technology world as Microsoft unveiled HoloLens and Microsoft’s use of holographic computing in the upcoming Windows 10 release. Holographic computing or augmented reality uses computer-generated images that are overlaid on real world videos. For example, a user can view a car through their smartphone. An app can project information such as make and model, fuel mileage, and other facts onto a real-time view of a particular car.
The federal government collects an amazing amount of economic data. Several agencies such as the Securities and Exchange Commission, the Treasury, and the Census Bureau collect economic data, ranging from the stock market activity to local business conditions. The Bureau of Labor Statistics (BLS) collects information on the labor market and is a rich source of data for researchers and the general public. The BLS offers two APIs for accessing labor data.
Recently, a reader pointed out that some of the APIs I write about are not really APIs but just datasets. Technically that is true but it only takes some development effort to turn a into an API. That is why I also highlight interesting federal datasets along with federal APIs. There are many federal datasets that should be APIs but how do agencies choose which datasets to build APIs?
According to some experts, over 80% of Americans will make a least one New Year’s resolution. There are the usual “lose weight,” “quit smoking,” or “exercise more” resolutions. Another popular set of resolutions involves learning new skills. So, if you are looking for a way to improve yourself while helping others, think about making a resolution to learn how to build a mobile app that can be used in disaster relief.
When websites were first created back in the 1990s, developers perfected their skills designing sites that presented content in an attractive and eye-catching manner. Content was completely contained within the four corners of the site. With the rise of Web 2.0, content creation became easier through blogs, wikis, and microblogging. Even so, content was still attached to that particular content creation tool. Content management systems (CMS) freed content from presentation.
We are in the middle of the holidays, and that means much driving to visit friends and relatives. I was just in Kentucky this past weekend where I spent a total of eight hours driving. I am sure many of you will spend even more time driving in the next three weeks. So, where do you find the best gas prices and how can you maximize your vehicle’s fuel mileage?
The Peace Corps just released a new dataset that lists all of the countries and regions Peace Corps volunteers serve. The API is RESTful and uses the JSON format. You have read in earlier columns about the different data formats for APIs and how to read the data presented by an API. As a refresher, I’ve created the following quiz based on the excellent documentation for the Peace Corps Countries and Regions API.
The federal government captures almost every economic data trend through several agencies. The Federal Reserve of St. Louis offers 238,000 economic trends through FRED® (Federal Reserve Economic Data). FRED® data can be accessed through the FRED® website or the FRED® mobile app (Android | Apple). FRED® data can even be pulled into Excel through a free plugin. Developers can take advantage of the vast data resources of FRED® and its cousin, ALFRED® (ArchivaL Federal Reserve Economic Data).
Every year, the Bureau of Justice Statistics surveys nearly 80,000 households and over 143,000 individuals about crime victimization. What is unique about this survey is that both reported and unreported crimes data is collected. The survey has a well-documented API which offers data in the CSV, XML, and JSON formats. Let’s examine the documentation to determine how a developer could use the data in the app. The National Crime Victimization Survey (NCVS) API is split into “personal victimization” data and “household victimization” data.
Back in 2000, I worked at a dot-com building website applications such as a real-time stock ticker ribbon and a real estate listings search engine. One of my favorite applications was an app for mobile phones. At that time, I used the Wireless Access Protocol (WAP), which displayed information using a special version of XHTML. Using the Kentucky Golf.com database, a user could search for information on a specific golf course by entering a ZIP code or using drop-down lists to search by county or course name.
Data.gov has 130,000+ datasets (as of November 3, 2014) many of which are designed for application developers. In previous columns, I’ve showcased some of the great applications built using federal APIs. Have you wondered where the idea for an app came from? Some developers start with an idea and then look for the API that best fits the idea. For example, a developer may want to create an app that alerts users of unsafe bus or limousine companies.
Today, refilling your medicine cabinet with bandages and over the counter medicine from your local drugstore may seem like a trivial task, but for Peace Corps volunteers working in remote villages around the world, this task can be much more challenging. As we take steps to forge a 21st century Peace Corps, such as dramatically reducing the time it takes to complete a volunteer application from eight hours to less than one hour, we are also looking into ways to tap the ingenuity of volunteer developers to support our Peace Corps volunteers abroad.
Thanks to the power of open data and APIs, federal agencies can now register their mobile native apps and websites on the Federal Mobile Products Registry and have them appear on the USA.gov Federal Mobile Apps Directory (formerly USA.gov Apps Gallery) almost immediately. When we launched the USA.gov Apps Gallery in 2010 there were less than 15 apps. To register an app, an agency would contact us with app info, download screenshots and create a “page” for that app.
Increasingly, we’ve noticed that our agency customers are publishing their highest quality images on social media and within database-driven multimedia galleries on their websites. These sources are curated, contain metadata, and have both thumbnails and full-size images. That’s a big improvement in quality over the images embedded within HTML pages on agencies’ websites. After some investigating, we decided we could leverage their Flickr and Instagram photos to build an image search engine that better met their needs.
It is fall when the weather becomes colder, and people start firing up their furnaces. While I was working on putting in more insulation and installing a programmable thermostat, I wondered if the federal government has an API to help me lower my utility bills. Yes, and it is a great API! The Department of Energy (through the Lawrence Berkley National Laboratory) has the Home Energy Saver API which is a comprehensive service to analyze home energy use.
Access to clean water is fast becoming a vital issue in the 21st century. Changing climate patterns are drying up aquifers and limiting the amount of water runoff from thawing snow packs. Drought conditions in California are effecting hydroelectric production while dry conditions in the West have increased the frequency and harmful effects of forest fire. Monitoring and mapping water conditions across the U.S. is a vital government service.
Content is no longer limited to .gov sites. As mentioned in a recent blog post, Sharing is Caring, Adding Social Media Accounts to Search, DigitalGov Search uses Flickr, Instagram, and YouTube to populate image and video search results. On September 30, 2014, I presented with Justin Herman from the Social Media Community of Practice about: What DigitalGov Search is How it integrates social image and video search How search analytics can help social media managers better understand their customers’ needs If you weren’t able to join us, you can download the slides or view the 30 minute webinar on YouTube.
The recent Ebola outbreaks demonstrate the need for current and authoritative health news. The Centers for Disease Control and Prevention (CDC) is the federal information source for Ebola and other infectious diseases, along with other public health data. Data.CDC.gov lists 48 datasets and views containing statistics from smoking to infectious diseases. Developers can use the Socrata Open Data API to pull JSON data into their apps. For those who are not developers, the CDC offers a way to embed health data into blogs, websites, and social media.
Tuesday, October 7 at 2pm ET, the National Library of Medicine at the National Institutes of Health will host a Google Hangout to talk about developing with the API, data, and open source code from Pillbox. I (Pillbox project manager), Mark Silverberg of Social Health Insights (builder of super cool health apps), and Maya Uppaluru of ONC (leads their Innovation Engagement program) will go under the hood of Pillbox.
Glad to be back after a three-week absence. I was preparing for the South Eastern Conference on Public Administration held in Atlanta this year. Great conversations and I can tell you that the academic community is hungry for more government data and APIs. A great example is this week’s API Briefing: the U.S. National Library of Medicine’s TOXNET Web Service. TOXNET lists 16 databases ranging from TOXNET, which maps chemical releases, to the Hazard Substances Data Bank (HSDB).
The National Archives and Records Administration and Wikimedia D.C., invite you to help us improve access to open government data on Wikipedia. We are excited to announce that we will be hosting the Open Government #WikiHack, a two-day hackathon at the National Archives Building in downtown D.C., over the weekend of September 27 and 28. Did you know that Wikipedia articles with NARA digital images saw over 1 billion page views in FY13 alone?
If you have ever been a caregiver for an elderly family member or friend, you know that there are many resources to help you in giving care. But finding these resources can be difficult and frustrating. The Administration on Aging in the Department of Health and Human Services (HHS) has been guiding people to local resources since 1991. Starting with a phone service, the Administration on Aging created its first website in 2001.
The National Aeronautics and Space Administration (NASA) has an enormous collection of aerospace and science data sets. NASA missions and projects can create amazing amounts of data. One example: the Earth Observing System Data and Information System has collected enough information to fill the Library of Congress (Data.NASA.gov). A more recent example: the Solar Dynamics Observatory receives 1.5 terabytes of data a day. As NASA admits, this much information can be overwhelming for agency API development.
As traffic to desktop .gov websites declines, how we publish our content increasingly matters. We need to meet people where they are as they seek information on the Internet. To do so, we need to adjust to the new world of mobile applications, social media, and instant answers provided by search engines. Freeing Content from Our Websites In this content sharing era, it is important to separate the content from how it appears on your site.
Last March, the openFDA team shared their still-in-progress API to potential users as part of the U.S. General Services Administration (GSA)’s API Usability Program. FDA created openFDA to allow researchers and developers to search their vast trove of public data, including information about adverse events (reports of undesirable experiences associated with the use of a medical product in a patient) submitted to the agency. The API Usability Program brings together developers from agency APIs and the private sector to evaluate how the API can be improved to be more user friendly.
Short URLs are useful for tracking clicks, but they can create a poor user experience because the person clicking the link can’t see the final destination. That’s why Go.USA.gov was created—to show users that they would reach official government information. To maintain this trust, Go.USA.gov is only open to government employees and only shortens government URLs—that is: .mil, .gov, .fed.us, .si.edu and .state.xx.us URLs. We are willing to make exceptions for other government URLs that meet our criteria.
How can you find the top 5 users of your open data? We were recently asked this question on the Open Data listserv, and while this information can be a good measure of success for open data programs, we also figured some of the answers shared would be of interest to the broader community. This blog post seeks to summarize and clarify those answers. What Defines a Top Third-Party Developer?
Content models provide an opportunity for agencies to structure, organize, distribute, and better publish information in multiple forms and on multiple platforms. Federal agencies discussed why content models are important for future-facing content in our What Structured Content Models Can Do For You Webinars in May and June. The point—with good content models, a single piece of Web content becomes an adaptive information asset that can be leveraged anytime, anywhere.
As the new school season approaches, it is a good time to see what federal datasets are available for educational app developers. Visit the developers’ page at ED.gov to find 36 educational datasets for educational levels. The datasets can be accessed in CSV, JSON, XML, and API formats. What is especially helpful is a PDF document that explains the data and the methodology behind the data collection. This is useful information for app developers when they combine datasets.
My name is Kin Lane, I am the API Evangelist, and a round two Presidential Innovation Fellow. I spend each day looking through the developer portals and hubs for API across the private sector. Recently I’ve also spent time looking at 77 federal government API developer portals and 190 APIs, and after all this review of some successful, and some not so successful APIs, you start to get an idea for what the minimum viable building blocks for an API hub in federal government should be.
Introduction Transparency in coding makes code more secure. Open-source development is development in the light, sometimes a harsh light, that shows every blemish. At 18F we strongly believe this improves the rapidity of our coding and the quality and security of the code. We keep the code open to each other, which allows us to quickly scrub in on projects and to dexterously apply the most talented resources to a problem without too much concern for who is formally working on or in charge of a given project.
Up till now, all the APIs that have been written about in The API Briefing were read-only APIs. That means that information is only one way: from the API to the user or app. These APIs do allow limited interactivity in that the database behind the API can be searched, but the existing data cannot be edited, or new data added to the database. There are some federal government APIs that are writable.
As federal agencies release APIs on an almost daily basis, keeping track of the numerous datasets has become a vital service for developers. The Department for Health and Human Services (HHS) manages HealthData.Gov which currently lists 1,680 datasets in 36 categories such as “Public Health,” “Health Care Cost,” and “Health Statistics.” To help developers find relevant datasets and keep up with newly-added datasets, the HealthData.gov API was created. Developers can use the Catalog API to search the catalog and receive meta-information about a dataset in the JSON format.
A growing trend both inside government and outside is to have a simple welcoming page for outside developers who may be interested in your team’s efforts. This material is often located at website.gov/developer 1 and points visitors to technical material that developers may be interested in, especially APIs. Collecting technical documentation in one place facilitates the developer experience, ensuring that they can find and begin using APIs with as little friction as possible.
Food deserts are areas where residents have little or no access to nutritional food. Food deserts exist because of low-incomes, lack of transportation, or too few stores that stock produce and other healthy food items. Governments from the local level to federal have implemented grant programs to encourage grocery store construction in the food deserts. Community activists have also worked to create food co-ops and encourage farmer markets to target the food deserts.
We recently released the first version of our API Standards—a set of recommendations and guidelines for API production. It is our intention that every 18F API meet these standards, to help us ensure a baseline quality and consistency across all APIs we offer now and in the future. These standards guide the user-facing implementation details of an API. Wherever possible, the standards prescribe a goal instead of a specific technology.
Once a federal agency releases an API, there are several ways they can be used in apps. The most common method is through hackathons. Hackathons are where an agency or agencies present the API(s) and invite developers to create prototype apps. The apps are then presented to subject matter experts for suggestions on creating the final app. There are many government hackathons on a variety of public issues. Visit Challenge.
In a recent event titled: “Intro to APIs: Working with URLs, JSON, APIs, and Open Data—Without Writing Any Code,” federal practitioners and supporters interested in open data attended an in-person workshop, led by Eric Mill, a key developer on GSA’s 18f team. This event was especially targeted to non-developers and explored the basics of APIs, using the Congress API, offered by the Sunlight Foundation, as an example. The purpose of the event was to showcase that anybody of any skill level can understand and use APIs without any coding knowledge!
The API Briefing: How APIs Provide Localized Information – NOAA’s Weather Service Data and FCC’s Broadband Services Map
The two featured APIs this week are excellent demonstrations of personalizing federal government data by where a user lives. Federal agencies collect a considerable amount of community data, from the Census Bureau’s surveys to the FDA’s local agricultural conditions. Thanks to GPS, app developers can locate a user’s immediate geographical location and tailor information based on the latitude and longitude coordinates. The National Oceanic and Atmospheric Administration has an API that provides current conditions and a four-day forecast by locating the nearest NOAA station to the user’s geographical coordinates.
This week, we will look at three different APIs that demonstrate how agencies use different technologies to serve out data. Presenting data in various formats encourages developers to build on federal APIs. As past columns have shown, the innovative apps created with federal data are quickly growing. The latest API news this week is how quickly the Department of Labor (DOL) built a Software Developer Kit (SDK) for Apple’s new programming language.
The Census Bureau recently released a “machine-readable dataset discovery service” that lists 41 Census data sets. It’s in spreadsheet form and gives a description of the datasets along with links to the API and developer documentation. What makes the discovery service machine readable is that’s based on Project Open Data’s “Common Core Metadata Schema” that uses a standard way to describe and index government information sources. The discovery service makes it easier for developers to find and mix different APIs together to create sophisticated apps.
Not sure how to get your datasets into Data.gov? We’ve put together an overview to show you how the process works. Agencies prepare their enterprise data inventories in data.json format and post them on their websites (agency.gov/data.json), pursuant to the Open Data Policy and following the guidance and using the tools available on Project Open Data. Data.gov also offers a tool called inventory.data.gov that can be used to assist agencies in creating their data inventories.
Not only does the Department of State have a great set of APIs, State also has an excellent example of how to build an informative and useful app. EducationUSA is a network of State Department advisers who help international students apply for U.S. university programs. The EducationUSA app has the most popular resources and services from the EducationUSA website, such as the ability to: Search for EducationUSA advising center information Follow the primary social media channels (Facebook, Twitter, & YouTube) View Frequently Asked Questions (in 8 languages) Discover new financial aid opportunities, and Utilize the Ask an Adviser (in five languages) function The EducationUSA app is an excellent example of designing for multiple-device experiences.
Developers, do you want to bring more detailed economic data to your next app? The Bureau of Economic Analysis (BEA) recently added several data sets to the application programming interface (API) we launched last year. The API now provides direct access to the gross domestic product (GDP) underlying detail tables. Those tables contain a wealth of statistics, including how much consumers spend on hundreds of items like furnishings, food and flowers and how much revenue the government takes in and spends.
The Food and Drug Administration (FDA) just released the OpenFDA Research Project. At the heart of the project is the OpenFDA API, which allows developers to perform searches on FDA’s drug information database. Coming soon is the ability to search FDA information on medical devices and information about food. Visit the FDA’s API Basics page to learn how to access OpenFDA including interactive sample queries. The FDA’s API documentation is a great example of how to create detailed guidance for developers.
Structured content and open content models can help you create content that is platform-agnostic, format-free, and device-independent. This was the theme of the “What Structured Content Can Do For You: Article Model” webinar last month. [youtube=http://www.youtube.com/watch?v=kG25vyQ5Jps&w=600] Using a content model is less about how you are crafting your message and more about how the internet is going to react to your content or how you can manipulate it, according to Holly Irving from the National Institutes of Health, Russell O’Neill from the General Services Administration, and Logan Powell from U.
Facebook is now the first social media platform to start verifying all federal government pages with their signature blue checkmark using the Federal Social Media Registry API. The Federal Social Media Registry provides the singular source that allows social media platforms to quickly collect real government accounts—emphasizing the critical need to ensure the trust, quality and security of citizen engagement. When the public searches for the new Central Intelligence Agency Facebook account, many different accounts pop up—but only one of them is managed by the actual CIA.
openFDA launched today and with it the first publicly available dataset—Drug Adverse Reaction and Medication Error Reports—that covers more than 4 million records from 2004 to 2013. The openFDA team compiled 10 things to help developers use this dataset more effectively, including: Interactive examples created by the openFDA team to give perspective to the data. The example scenarios with queries help teach how to perform different types of searches on the data.
The Food and Drug Administration collects drug labeling information for human prescription, over-the-counter, homeopathic, and veterinary products through a special markup language called “Structured Product Labeling” (SPL). The database created from the SPL submissions is a treasure trove of health information that is valuable to pharmacists, doctors, and the ordinary health consumer. The problem is that data is hard for developers to access and process. Until recently, when the National Library of Medicine released open source code for “Pillbox.
In our final video interview with Sarah Crane of USA.gov, she talks about adaptive content and how it works with APIs. Missed Part 1 and Part 2? Watch them to find out how USA.gov dealt with their inconsistent customer experience and content sprawl. [youtube=http://www.youtube.com/watch?v=giK-RsHjA4c&w=600] Interested in learning more about adaptive content and content modeling? Check out the new Structured Content Models and the training on Event Model Creation. We’ve created an editorial theme calendar to coordinate content each month around one focus.
Federal employee training is about to receive a much-needed boost in the President’s 2015 Budget Request. Training is essential to the federal workforce and agencies have a number of learning management systems to deliver online training along with the traditional classroom training. The problem is that all of these training sources don’t share information with each other about what training a learner has completed. Compiling a training record is a tedious and mostly manual process of printing out certificates, filling out SF-182s, and keeping paper records.
In May 2009, Data.gov was an experiment. There were questions: would people use the data? would agencies share the data? and would it make a difference? We’ve all come a long, long way to answering those questions, starting with only 47 datasets and having 105,000 datasets today. We realized that this was never simply about opening up government data, but rather about growing and nurturing an open data ecosystem to improve the lives of citizens.
Over the past year, a GSA collaboration has seen a project that offers API Usability Testing to federal agencies go from the pilot stage to a regular, robust series. Already, 13 agencies and programs have participated, and several more participate with every monthly session that passes. The best examples from across the government have made clear that one of the most important tasks of API producers is to regularly engage their developer community and listen to what they have to say.
Part 2 of our interview with Sarah Crane from USA.gov shares how the USA.gov team is tackling content sprawl with the USA.gov API. Missed Part 1 last week? You’ll want to watch it before next Tuesday, when we will publish Part 3 where Sarah talks about adaptive content. [youtube=http://www.youtube.com/watch?v=PtXsAIoRqb4&w=600]
We won’t build the government of the 21st century by drawing within the lines. We don’t have to tell you the hard work of building a digital government doesn’t exist in a vacuum or a bubble. Show us social media without mobile, Web without data and user experience without APIs. You can’t? That’s right—in reality, digital government intersects and cuts across boundaries every day in order to deliver the digital goods.
Around the D.C. area, one of the first signs of spring are the numerous farmers markets. In my neighborhood alone, I regularly visit four farmers markets that have a wide variety of produce and baked goods. Farmers markets are good for the local economy, and the easy access to fresh fruits and vegetables helps local communities. Realizing the importance of farmers markets, the USDA released the Farmers Market Directory API so that developers can create apps to help people find farmers markets in their area.
APIs and User Experience go together like gummi bears and ice cream. An API is a product just like a car, a website or a ballpoint pen. It’s designed to help someone do something. Products are either designed well—they meet expectations and deliver value—or they are designed poorly and create frustration and confusion. Inevitably, bad products are abandoned without a thought, like an old T-shirt with holes in it.
As a “warfighting-ready force,” the U.S. Navy can bring it wherever and whenever it’s needed around the globe. Now this armed force is bringing it—the latest news, video, photos, multimedia, special events and more—to sailors and their families through a slick new app that effortlessly crosses borders and software platforms. The official U.S. Navy app curates content from a variety of Navy assets and offers users a mobile experience that includes:
At DigitalGov Search, we keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We recently came across Gov.UK’s philosophy on and approach to coding in the open. It caught our attention and we realized we should also articulate our open source strategy. Use and Contribute to Open Source Projects Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites.
Welcome to the new home of openFDA! We are incredibly excited to see so much interest in our work and hope that this site can be a valuable resource to those wishing to use public FDA data in both the public and private sector to spur innovation, further regulatory or scientific missions, educate the public, and save lives. Through openFDA, developers and researchers will have easy access to high-value FDA public data through RESTful APIs and structured file downloads.
If you have ever visited census.gov, you know that sorting through the vast array of information about America’s people, places and economy can be daunting. Based on customer research and feedback we collected and analyzed over time, we heard loud and clear that both search and navigation of our site could be much better. Visitors to census.gov should not have to work so hard to find the information and statistics they are looking for to complete their research, personal projects or business needs.
Improving the federal government’s ability to deliver digital information anytime, anywhere, on any device—via open content—is a key goal of the Digital Government Strategy. A content management system (CMS) can help your agency move to an open content model, making it easier for people to find, share, use, and re-use your information. The key steps in getting ready to move to a CMS include: Prepare Your Content Choose a CMS Migrate Your Content to a CMS Prepare Your Content Develop a Content Strategy A content strategy defines such things as topics, themes and purpose, and can also play a part in website governance, customer experience, metadata and search engine optimization (SEO).
Like website development, API security revolves around three stages—planning the API, testing the API, and monitoring the API after it has launched. The planning stage requires those involved to conceptually map several design decisions and the impact that they will have on security. The second stage applies your agency’s security program to the API release candidate. Lastly, the third step integrates your API in your agency’s continuous monitoring frameworks.
These are the elements you should include in your federal API release. Homepage Each of your public APIs needs a page to serve as a hub to provide access to all information and tools associated with it. By using the page’s sidebar, footer, and sub pages, you can directly include or link to each of the following components that exist for the API. This allows for ready discovery of anything a potential developer may need, minimizing the effort that is asked of them and maximizing the adoption.
“APIs for WhiteHouse.gov? What in the heck does that mean?” Like President Obama, you may be asking the same question. DigitalGov University set out to answer that question during its API Standards from the White House webinar with Leigh Heyman and Bryan Hirsch, from the White House Office of Digital Strategy. [youtube=http://www.youtube.com/watch?v=-bJo6zii7lw&w=600] After launching the We The People petition site, citizens using the tool were asking how they could improve their own engagement levels on the petitions and wanted to look at the data.
What Are APIs? An Application Programming Interface, or API, is a set of software instructions and standards that allows machine to machine communication—like when a website uses a widget to share a link on Twitter or Facebook. [youtube http://www.youtube.com/watch?v=BVeiRCEwJx8] [Extended Version] When we talk about APIs we are referring to Web services or Web APIs. This aligns with the current trends to use Web APIs to support sharing content and data between communities and applications.
One way agencies can offer APIs for their data is to use the built–in functionality of Data.gov. The information that is hosted as interactive datasets have an API layer which agencies can make available through documentation in the developer’s section of the agency’s website. The guide below will help you do this. Process Upload a dataset as an interactive dataset in Data.gov. A. Your agency should have a Data.
Good APIs can transform intimidating data sets into something that people can use. Good government APIs can create a better connection between government and citizens. The Digital Government Strategy has spurred Federal adoption of APIs, and the Digital Services Innovation Center and DATA.GOV teams are supporting these efforts by releasing a swath of guidance and resources, hosting online education series and launching a number of hosted tools at Labs.Data.gov.
Similar to website analytics, API analytics focus on reliably reporting the metrics which are most useful to its stakeholders. There are a many ways of collecting, reporting, and consuming API analytics but all revolve around the industry–accepted norm that some form of analytics are crucial to any API program. The most basic metrics will track the number of API registrations and the number of API requests, but as important as understanding how much an API is used is understanding how it is used.
Design At their core, developers want APIs for very straightforward, pragmatic ends. You should always design your APIs and document them with the goal of making it easier for developers to use them. Doing so results in greater adoption and a healthier, more successful API. The least efficient way to support developers would be to work with each interested developer individually on any question or problem that they have.
After choosing a set of information or services to offer via API, some of your next steps are to plan and implement the API. You’ll still need to prepare documentation, tools, and other elements that make a complete package for the API, but at the center is the actual Web service itself. In many situations, existing IT resources or the current system operators handle this step. There are several options at this stage, each with advantages and disadvantages.
Hosted API Tools Labs.Data.gov is a repository of shared services to prototype and provide developer resources to government agencies. Each tool uses Web services and lightweight, open source code to provide powerful functionality. Agencies are encouraged to improve any project and submit pull requests in order to share the improvements with others. API Standards Template With the open source release of the White House’s API Standards template, agencies have a complete model for API design and best practices that includes the best practices and agreed–upon norms of the developer community.
Audit Research existing APIs Regardless of your agency’s level of progress in API production, your first step is to create a developer hub that links to any of your agency’s existing APIs. This can help identify current efforts and connect you with others in your agency already working on APIs. Follow up with a deeper scan for APIs that your agency may already be publishing:
Common Technical Choices Protocol API protocol is the set of rules that govern how an API functions. The protocol outlines the structure and definitions of the API. The two common forms are REST and SOAP. REST is the dominant choice for API protocol because it uses the http protocol that powers the Web. REST supports more data formats, requires much simpler documentation, has better performance, can be cached, and is faster to use.
Understanding the benefits of API production allows you to coordinate with system owners and other stakeholders to modernize the agency’s systems and unlock the sizable potential. Here are just some of these opportunities. Efficiency Providing API access allows for content to be created once and automatically published or made available to many channels. Your agency’s content is ready for easy sharing and redistribution to deliver your mission directly to more citizens.
Agencies have been working away at building better digital services and here, at the Digital Services Innovation Center, we’ve been building resources to help. We have been focusing on three areas, The Digital Analytics Program. We announced this program in early October to help agencies better measure performance and customer satisfaction to improve service delivery. It includes digital metrics guidance and best practices, training and a federal-wide Web analytics tool and support.
A case study on how NASA is choosing a new enterprise content management system (CMS). The Challenge NASA.gov needs a new enterprise CMS. They’re facing issues such as software obsolescence, inconsistent website governance, and a large amount of unstructured content stored in flat HTML files. Their current system is almost a decade old, and the vendor no longer provides technical support. They need an enterprise solution that will enable offices throughout NASA to collaborate on content creation, instead of having each component create content in isolation.
_ Mobile Gov Experiences are agency stories about creating anytime, anywhere, any device government services and info. This entry is a story shared by Centers for Disease Control and Prevention._ The Centers for Disease Control and Prevention (CDC) uses content syndication to share important health information with a variety of federal public health agencies, state and local public health departments, non-profit organizations, academic institutions, and commercial organizations. Why We Did It CDC developed content syndication to give our public health partners and other interested parties the tools to deliver credible content directly to their visitors.
There’s an easier way to get content and data into the hands of citizens. Application Programming Interfaces, or APIs, are web services that allow people to more easily consume content and data in multiple ways–via mobile devices, mobile apps, innovative mash-ups, and much more. Simply put, “APIs are a better way to get government information and services into the hands of the people who need them.” To help agencies better understand APIs, DigitalGov University hosted a webinar, An Introduction to APIs, with experts from NASA and CDC.
_ Mobile Gov Experiences are agency stories about creating anytime, anywhere, any device government services and info. This entry is a story shared by Healthfinder.gov_. The actual healthfinder.gov site was launched approximately twelve years ago, while the API is a fairly recent development that has occurred in the past year and a half. Why We Did It The reason to go with an API rather than an app or another any other format was in large part due to the type of information we were providing to our audience.