The U.S. federal government is probably the one of the biggest (if not the biggest) producers of data. Every day, thousands of federal workers collect, create, analyze, and distribute massive amounts of data from weather forecasts to economic indicators to health statistics. Federal government data is a major driver of the American economy as businesses use the data to make decisions or blend the government data into products and services sold to consumers.
The best way to learn a new technical skill is to just play around with the technology. Learning through playing with technology goes for building websites, mobile apps, and now, chatbots. As chatbots have become more popular, some online sites will let you create a chatbot with little or no programming. Now, realize that the easier it is to create the chatbot, the less sophisticated the chatbot will be. However, you may not need a sophisticated chatbot that can handle almost any situation.
A Washington, D.C. think tank recently released reports advocating using artificial intelligence (AI) tools to reorganize the federal government. There has been a larger debate about the effects of automation on the private sector and the American economy, but this appears to be one of the few reports focusing on the federal government. According to the think tank, the U.S. government “could yield $23.9 billion in reduced personnel costs and a reduction in the size of the federal workforce by 288,000.
Over a year ago, I wrote about the potential of new chatbot blockchain digital autonomous organizations. I was excited about the possibilities of how the emerging technologies of chatbots and blockchains would merge to create the digital autonomous organizations and what this could mean for delivering government services. Since then, 2017 has being called the “Year of the Chatbot” because of the rapid advances in artificial intelligence (AI) and the explosion of tools that make it easy to create chatbots.
Learning—and practicing—five imperatives of network leadership can help agencies improve their odds for successful digital transformation. Many organizations are undergoing digital transformation because the organizations see that it is necessary for long-term survival. However, digitally reinventing the organization is “one of the hardest journeys to make.” According to industry experts, two-thirds of organizations will fail at digital transformation. With these dismal odds, what can agency leaders do to improve their agency’s chances in successfully digitally transforming?
When I was in the private sector, around the year 2000, I worked for an information technology (IT) consulting company as a project manager and developer. On one project, I provided support for early mobile devices given to medical students. I worked in a small office around the corner from the cardio-respiratory simulator (CRS). The CRS was a life-sized human dummy that could simulate several conditions including a heart attack, a collapsed lung, and other heart and lung issues.
Since 2007, a major consulting firm has conducted an annual survey on organizations’ “Digital IQ.” In the ten years of organizations grappling with digital transformation, what has been learned? From the report: Focus on the human experience [emphasis in the original]: Rethink how you define and deliver digital initiatives, consider employee and customer interactions at every step of the way, invest in creating a culture of tech innovation and adoption, and much more.
In the last national election, the earliest born members of Generation Z voted for the first time. In 2019, the American workforce will see the influx of tens of millions of Gen Zers who, according to some researchers, will be a stark contrast to the Millennials that will make the largest part of the 2020 workforce. According to one researcher’s study of Gen Z, this generational group has seven distinguishing traits:
The first chatbot, ELIZA, was created back in 1964 to demonstrate that communication between humans and computers would be superficial. However, much to Dr. Weizenbaum’s (ELIZA’a creator) surprise, people easily formed friendly relationships with the computer program. People forming relationships with ELIZA was especially surprising considering just how simple the program was regarding generating conversational responses. ELIZA essentially parroted back what the users typed but, this was enough to convince people that the program seemed to care about the person.
It’s been a while since I’ve checked in on enterprise architecture (EA). My last in-depth work with EA was around 2011 when I was on detail to the Office of Personnel Management’s Open Government Team. The EA model I worked with was the top-down organizational design of information technology assets, data assets, and business processes. Many of you are probably familiar with this traditional EA model. Six years later, it is predicted that in 2018 that “half of enterprise architecture (EA) business architecture initiatives will focus on defining and enabling digital business platform strategies.
Forbes magazine recently ran an article showcasing six handy mobile apps that were built using federal government open data. The apps range from the Alternative Fueling Station Locator to ZocDoc (a doctor locator). What I especially like about the Forbes article is that the author describes the federal government data sets behind each app. There are many more mobile apps built by federal government agencies or using federal government data sources.
The Department of Education (ED) launched its first developer site. The developer site is built on GitHub which will make it easier for ED to centralize their code and Application Programming Interfaces (APIs). Currently, ten APIs are on the developer site: The Civil Right Data Collection (CRDC) APIs: These three APIs give information on public school enrollment in 2013–14, chronic absenteeism in 2013–14, and out-of-school suspension in 2013–14. The College Scorecard API: This is data from the College Scorecard project which allows student and families to “compare college costs and outcomes as they weigh the tradeoffs of different colleges, accounting for their own needs and educational goals.
You may have heard of “serverless architecture” or Amazon Web Services (AWS) Lambda product and wondered what is unique about this new buzzword. As with many new digital cloud technologies, serverless architecture could mean two things. It may be applications that are built using third-party cloud applications. Or serverless architectures could be pieces of code that live in the cloud and only run when called on by a user: event-driven functions.
This week, I want to briefly discuss the human resources challenges in finding the new IT technology workers for the government. As agencies move toward microservices, artificial intelligence chatbots, and deep learning application programming interfaces (APIs), the demand for experts in these fields continues to grow fast. The universities and professional development programs are not churning out the talent fast enough while governments are competing with private industry for what experts are currently available.
December 9, 2016, will be the 110th anniversary of Admiral Grace Hopper’s birthday. Admiral Hopper was a pioneer in computer programming who created the first compiler and whose ideas lead to the creation of COBOL. An apocryphal legend also credits Admiral Hopper with coining the terms “bug” and “debugging.” The GSA’s IT Digital Service Team will celebrate Admiral Hopper’s birthday with a beginner-friendly hackathon. The Grace Hopper Day Hackathon is the perfect hackathon for beginners.
The open source movement has changed how we develop software, create content, and even do science. Using a community to help complete projects and bring about change has become so ubiquitous in the last ten years that it has even earned a name — crowdsourcing. Crowdsourcing is a powerful tool and is now being used to help organizations digitally transform. “Driving successful change in a large organization has always been one of the most difficult activities in business… .
As the Federal government agencies begin the digital transformation journey, becoming a data-driven organization is even more vital. What does it mean to become a data-driven organization? According to one definition, “[a] data-driven company is an organization where every person who can use data to make better decisions, has access to the data they need when they need it.” There are many theories are on how to create a data-driven organization, but few case studies that demonstrate the actual process.
A recent study of big data initiatives in 65 cities has interesting guidance for Federal big data initiatives. The researchers studied how data is collected and then used for decision making in what they called “the framework for Big Data initiatives.” There are two major cycles in the framework: “The data cycle governs the tools and processes used to collect, verify, and integrate data from multiple sources. Because of the variety of data sources involved, data teams in this cycle are [sic] often composed of representatives from multiple departments to leverage their field expertise and insider understanding of the data.
In December, I plan to write two postings detailing a scenario analysis for the next ten years of the Federal government’s data technologies. Governments are on the cusp of amazing technological advances propelled by artificial intelligence, blockchain technologies, and the Internet of Things. Also, governments will face new challenges such as the recent global cyber attack that took down Twitter and Netflix. I want to invite you, the reader, to also send in your predictions for the future of Federal government data.
The Data Briefing: Using Artificial Intelligence to Augment the Work of Frontline Government Employees
You have probably read about the recent release of the White House’s report on using artificial intelligence (AI). As with previous technologies, AI holds much promise in the areas of education, commerce, criminal justice, the environment—almost all aspects of the American public’s life. AI also poses a danger if it is not properly managed and controlled. This is why the report advises that “[a]s the technology of AI continues to develop, practitioners must ensure that AI-enabled systems are governable; that they are open, transparent, and understandable; that they can work effectively with people; and that their operation will remain consistent with human values and aspirations.
I recently sat down with Michelle Earley, Program Manager, to discuss the new changes for the 20th anniversary of USAJOBS. 1) What are the three big lessons learned from 20 years of building and managing USAJOBS? I think one of the greatest benefits of being an Agile program is that we are constantly learning. In 2013, our team implemented the first phase of the data warehouse which provided agencies with data that could be leveraged to improve recruiting efforts.
The Data Briefing: The Federal Data Cabinet—Promoting Data Literacy, Cultural Change, and the Federal Data Applications Ecosystem
Last Wednesday, the White House held the first Open Data Summit to showcase the open data accomplishments of the Obama Administration. One of the highlights was the formation of a government-wide “data cabinet.” Announced by Chief Data Scientist DJ Patil, the data cabinet is essentially a community of practice comprising the Federal agency’s data professionals. As Dr. Patil explains, the real issues concerning technical projects revolve around cultural issues. I couldn’t agree more.
One day, at an unnamed agency, the Outlook server crashed. The server stayed down for the rest of the afternoon. Deprived of email and meeting calendars, employees wandered around trying to remember what meetings they had to attend. Other employees went searching for people who they ordinarily would email. There was confusion that made people realize just how dependent they were on a single software program. As the Federal government moves toward digital transformation, I have been thinking about how agencies can best weather the transition from legacy systems to cloud-based, agile applications.
The United Kingdom’s (UK) Digital Service has researched ways to increase data science literacy among the UK public service. Data science literacy goes further than data literacy, in that civil servants will know how to apply data science concepts and methods in their everyday work. I thought it would be useful to share the UK Digital Service’s findings to help federal government employees develop their data science literacy. Before discussing how to increase the data science skills of federal employees, let’s discuss why.
The Data Briefing: Harnessing the Internet of Things and Synthetic Data to Provide Better Flood Warnings and Prevent Veterans Suicides
Two significant items in federal government data in the last few weeks: The Department of Commerce releases the National Water Model. The National Water Model provides a comprehensive model of river flows so local communities can better prepare for possible flooding events. What is especially amazing about the National Water Model is that it pulls data from over 8,000 stream gauges. Stream gauges are automated measuring stations that measure water flow, height, surface runoff, and other hydrological data.
The Data Briefing: Microservices and Serverless Apps — A New Direction for Federal Government Mobile Apps?
Continuing from last week’s column on DevOps and containers, I will explain two other hot trends in IT — microservices and serverless apps. For those who want official federal government guidance, the National Institute of Standards and Technology (NIST) has released a draft special publication on microservices, application containers, and system virtual machines (PDF, 660 kb, 12 pages, February 2016). I wrote about microservices and containers in February 2015 as two API* trends to watch.
The Data Briefing: DevOps and Containers and Why They Are Important to Transforming Federal Government IT
You may have heard about “DevOps” in the news or when meeting with IT professionals. What exactly is DevOps and what, if any, connection does it have with agile? Also, what do “containers” have to do with all of this? In this week’s column, I will introduce DevOps and a related technology: containers. Some DevOps practitioners will argue with my interpretations, and I invite these practitioners to explain their perspectives in the comments.
The Data Briefing: FINDing Great Global Development Data Visualizations Courtesy of the State Department
Federal agencies have been releasing some fascinating data visualization tools in the last year. Recently, the State Department unveiled the Beta version of FIND or the “F Interagency Network Databank.” From the description in the FAQ: “The F Interagency Network Databank (FIND) is an online tool that enables users to explore and analyze national level data, and then share what they discover. FIND was designed around the needs of U.
The Census Bureau conducts more surveys than just the Constitutionally-mandated Decennial Census. There is also the American Community Survey, the Economic Census, the County Business Patterns series, statistics on Nonemployer businesses, and the Survey of Business Owners and Self-Employed Persons. On their own, each survey is full of useful information for researchers, local and state governments, and entrepreneurs. However, how valuable would the data be if it were mixed and displayed geographically?
While you are outside hunting Pokemon or helping your children hunt Pokemon, consider adding another mobile app to your smartphone or tablet. The Federal Emergency Management Agency (FEMA) mobile app alerts you about severe weather and other natural disasters. The app is also a great information resource on surviving disasters and connects you to FEMA for immediate assistance. A feature you won’t find in many other apps is the ability for users to crowdsource photos of disaster areas to help first responders.
The debate between responsive websites and mobile apps took a decisive turn this week when the United Kingdom’s Digital Service (UKDS) banned the creation of mobile apps. In an interview with GovInsider, the founder of UKDS, Ben Terrett, explained that mobile apps were too expensive to build and maintain. Responsive websites were easier to build and updating the application only requires changing one platform. “For government services that we were providing, the web is a far far better way… and still works on mobile,” Terrett said.
It is at the intersections of fields where you find the most fascinating and innovative concepts. Recently, a conference on “Open Human Resources and the Cognitive Era” explored the use of chatbots and blockchain technologies in human resources. Human Resources (HR) is quietly undergoing a revolution as many HR practitioners are transforming HR by using open source concepts. It is fascinating to see how cognitive technologies and cloud technologies are changing HR from a transactional and compliance function to an essential strategic organizational asset.
Business processes have fascinated me since I took an undergraduate philosophy course in modern business management. A part-time professor who was a management consultant by day taught this unusual class. Perhaps business management thinking was first experimenting with ideas that would later lead to the agile and lean movement today. From this class I learned that nearly all organizational issues could be traced back to bad processes rather than poor workers.
Few other federal agencies deal with as much data as the National Aeronautics and Space Administration (NASA). Big science creates big data, and NASA manages many of the biggest science projects in world history. Even in the early days of NASA’s history, NASA pioneered new ways to create and store data. So, in the world of the cloud, Internet of Things, and intelligent agents, how does NASA deal with its big data needs?
Cognitive computing has been receiving a good deal of attention lately as more companies have been building intelligent agents. Ever since IBM Watson’s 2011 appearance on Jeopardy, cognitive computing has spread into healthcare, investing and even veterinary medicine. However, it is only recently that cognitive computing has spread into government applications. As the name implies, cognitive computing is where computers operate much like the way people think. Computers use data mining techniques, pattern recognition algorithms and natural language processing to search a large set of unstructured data to find solutions.
My first column when I came back from last year’s summer sabbatical was on the U.S. Patent and Trademark Office’s (USPTO) PatentsView project. PatentsView became one of the Department of Commerce’s most viewed apps in 2015. Building on this success, USPTO released a beta version of its open data portal. The USPTO open data portal is divided into four different sections. The first section leads to patent and trademark datasets.
USAGov recently released a list of six great federal government mobile apps. There were many apps released by the federal government over the last 5-6 years on a wide range of topics and services. Many are well-designed and useful to the American public. So, what are the outstanding federal government apps for 2016? The Department of State’s Smart Traveler. First launched in 2011, this mobile app helps international travelers find U.
Last month, I worked to create a “Citizen Science Passport” for the federal agencies participating in the USA Science and Engineering Festival. Seven federal agencies offered some form of crowdsourcing or citizen science activity at their booths such as the U.S. Department of Agriculture’s exhibit on food safety or Environmental Protection Agency’s build-your-own air monitoring kit. Attendees would participate in each of the agency’s citizen science activity to receive a stamp on their passport.
There are many scary tales in the world of knowledge management and data management. Tales of missing data that was lost through the administrative cracks, such as the story of the missing Apollo 11 moonwalk tapes that most likely were erased by accident. Or the 36-year search for the original Wright Brothers’ patent, which was happily re-discovered this month. As more data is being created at ever-increasing speed and complexity, there will be more missing data horror stories.
Ten months ago, I wrote about the rise of the post-app world in which mobile personal assistants would do the work of five to 10 apps combined. These mobile personal assistants, now known as chatbots, would work through conversational interfaces (voice and instant messaging, for example). The idea is to build more natural interfaces for people to access information services and perform complicated online tasks. Facebook has now joined in the new conversational commerce marketspace along with Google and Apple.
It has been over seven years since President Obama signed the executive order that launched the federal open data movement. Much progress has been made, and there is still more to do. Along with the United States, over 100 nations have started programs to provide open access to government data. From large metropolitan governments to small cities, governments are opening up their data to provide better transparency and better delivery of government services.
The Office of Personnel Management released a new look and functionality to USAJOBS in February. I recently contacted Michelle Earley, the USAJOBS Program Manager, to ask about the changes to USAJOBS and the data it provides. 1. What are the priorities this year for the USAJOBS team and the site? “The priorities for this year include: Unifying the experience Incorporating a comprehensive content strategy to transform the readability of the website Improving the Job Opportunity Announcement (Represents the agency) Improving the User Profile (Represents the job seeker/applicant) Improving Search, which is the mechanism that brings together the job seekers and agencies USAJOBS hopes to continue to act as a trusted public service career platform that creates a responsive and transparent experience for its users.
Three recent stories demonstrate how opening up federal government data and using agile methods to create federal government software can spur innovation while saving tax money and helping the American public. In its Second Open Government National Action Plan (PDF, 639 KB, 5 pages, September 2014), the White House called for a government-wide policy on open source software. Recently, the Office of Management and Budget released a draft policy “to improve the way custom-developed government code is acquired and distributed moving forward.
Open data and APIs* have not only transformed the federal government; open data and APIs are also transforming tribal, state and local governments. Like federal agencies, some tribal, state and local governments are ahead of other governments in open data innovations. This situation reminds me of my earlier work with the Office of Intergovernmental Affairs in the General Services Administration. In 1998, I was a Presidential Management Fellow working on a project to catalog how state and local governments were using websites to deliver government information and services.
The increasing sophistication of mobile devices has created many opportunities for developers. Thanks to APIs* and open data, developers can build thousands of mobile apps and mobile websites to meet users’ needs. This opportunity has created one of the most contentious debates in the mobile development community: mobile apps versus mobile websites? There is, yet, no solution to the debate. However, there are some advantages and disadvantages to both types of mobile solutions.
Citizen developers are people who do not work in information technology (IT) but have built IT applications. Back in the mid-80s, business people would smuggle in personal computers to run their spreadsheets and word processing applications (anyone remember VisiCalc and Bank Street Writer?) instead of having to rely on data processing departments. Today, citizen developers use no-code or low-code services such as IFTTT (If This Then That) or QuickBase to build their business apps.
Algorithms are becoming more important as the amount of data grows, and the complexity of government and business processes grows. Put simply, an algorithm is just a set of steps for solving a problem. If you shop online, use an online social network or a mobile app to plan your route, then you are using an algorithm: A sophisticated algorithm that uses large amounts of data to make hundreds (or thousands) of decisions in milliseconds.
For many agencies, what data to make open is left up to the agency’s judgment. This has worked well as agencies do a good job in understanding the public’s needs for specific datasets. Even so, as developers and citizens begin using the open datasets, there is increasing demand for specific agency datasets. The issue is how to best accommodate those requests given the constraints of agency budgets and open data support staff.
The Congressional Research Service recently released a report (PDF, 688 kb, 17 pages, January 2016) describing the big data ecosystem for U.S. agriculture. The purpose of the report was to understand the federal government’s role in emerging big data sources and technologies involved in U.S. agriculture. As the report author, Megan Stubbs, points out, there is not even a standard definition of big data. “Big data may significantly affect many aspects of the agricultural industry although the full extent and nature of its eventual impacts remain uncertain.
Federal agencies are doing well in fulfilling the 2012 Digital Government Strategy by providing numerous mobile apps for American citizens. According to a report from IBM’s Center for the Business of Government, 76 federal agencies have at least one mobile app. As of July 2015, there are nearly 300 federal government mobile apps that provide at least one of the following: General information and news services Client services such as providing and processing government forms Crowdsourcing Health and safety information Educational services According to the report, mobile devices were one-third of the traffic to government websites, as of July 2015.
Many IT pundits predict 2016 will be a major tipping point in data and related technologies. Here are just a few predictions: 1) The Internet of Things—The number of devices that can connect to the Internet increases, especially in consumer electronics. Also, the number of sensors will dramatically increase providing more real-time data on weather, electrical power usage, and similar data. The number of devices connected to the Internet is projected to exceed the number of human Internet visitors.
Some highlights from the recent fall conference of the Association for Public Policy Analysis and Management: Localities that receive disaster mitigation funds also have more disaster declarations. Longer, more detailed proposed regulations receive fewer challenges to implementing the regulation. Agencies that are better at quantifying their results are safer from budget cuts. The findings above were all based on ready access to open government data. In fact, many more public policy and administration scholars can do more detailed, and innovative research thanks to federal agencies’ release of these datasets.
Recently, DigitalGov devoted an entire month to exploring how good user experience (UX) helps government design better digital products and services. UX is the art and science of understanding how people will use a website or mobile app to solve a problem or meet a need. UX is a combination of neuroscience, communication theory, information architecture, content strategy, graphic design, and responsive programming to build an experience that is inviting and beneficial to users.
The Smithsonian Institution’s Air and Space Museum just released a new educational mobile app, Mobile Missions. From the website: “Find out if you are cut out for a career in aerospace with our free mobile app, Mobile Missions. Take our quiz to discover the best aerospace career for you. Explore objects from our collection related to your chosen profession. Answer challenge questions to receive in-app badges and rewards. Document your journey by inserting your selfie into a historical image related to your aerospace career and share with friends.
Pop quiz on statistics and data science (answers at the end of the article): 1) I have some data on accidents at railroad crossings. One variable indicates the compass direction a railroad crossing faces (North, Northwest, Northeast, and so on). This variable is a/an: Ordinal Categorical Directional Interval 2) I have some ordinal data that I want to analyze for trends.
A month ago, I wrote about the White House’s call for data scientists and app developers to come together to help combat suicide. On December 12, 2015, there will be five hackathons around the U.S. to #HackSuicide. All the hackathons are free and open to the public. Even if you are not a data scientist, app developer or mental health expert, you may want to attend one of the events to learn how data can be used to solve a vital public health issue.
Some of you may remember when President Reagan opened America’s Global Positioning System (GPS) data. President Reagan gave all countries access to the GPS data in response to the Soviet Union shooting down Korean Airlines Flight 007 on September 1, 1983. I do not believe that the U.S. realized how much opening up GPS data would revolutionize the world economy, health services, travel and almost every other aspect of daily life.
Standing on the corner, waiting in the rain, I swear I’ll never, ever, use that app again. Why? Because the bad user experience (UX) design was preventing me from determining when the Metrobus would arrive. UX is everything from the visual design to the navigation structure of the website or mobile app. This month, DigitalGov is focusing on UX design. Good UX design is based on understanding how people perceive and process information on everything from websites to mobile apps.
By the time this is published, the United States, along with 160 other countries, will be celebrating Global Entrepreneurship Week (November 16th through November 22nd). November is also National Entrepreneurship Month with November 17th being National Entrepreneurs’ Day. As President Obama stated in his proclamation: “In keeping with our goal of fostering economic growth through private-sector collaboration, the federal government is accelerating the movement of new technologies from the laboratory to the marketplace, increasing access to research awards for small businesses, making more data open to the public [emphasis mine] and catalyzing new industry partnerships in critical fields such as advanced manufacturing and clean energy.
I (virtually) attended the Third Annual Safety Datapalooza last Thursday and was greatly impressed by the projects and initiatives for public safety. This was a great event, and I am glad that live streaming was provided for those who could not attend in person but have a great interest in using government data for disaster preparedness. If you have not already visited disasters.data.gov, please do. It is a great portal for data, apps, and tools for developers who want to help build vitally-needed public safety resources.
Have you worked with an employee with a disability? Are you an employee with a disability? Then, you know the unique challenges of the average workplace that able-bodied colleagues may never experience. Workplace challenges could be overcome with accommodations such as larger computer monitor displays, wheelchair-accessible office furniture or a voice reader. In some cases, a mobile app is a solution to a workplace challenge. October is National Disability Employment Awareness Month.
DigitalGov’s theme this month is mobile moments, which explores the impact of mobile applications in the federal government. For this post, I am examining the more than 300 mobile apps created by the federal government. An updated list of federal mobile apps is on USA.gov. According to the list, 73 federal organizations have released mobile apps on a wide variety of topics. The National Oceanic and Atmospheric Administration has the most mobile apps with 31 releases.
The Data Briefing: White House Asks Data Scientists and App Developers to Help Suicide Prevention Efforts
The White House issued a call on September 30, 2015, for data scientists and app developers to help with a vital public health issue: suicide prevention. From the official announcement: “If you are a data scientist, analyst, tech innovator, or entrepreneur interested in sharing ideas and resources for suicide prevention, we want to hear from you! Please send a brief note about your ideas and resources to mbasco[at]ostp.
OpenNASA has recently completed another redesign of their site. With over 31,000 data sets, 194 code repositories and 36 APIs, OpenNASA probably has the largest collection of open data of any of the federal agencies. An especially helpful feature is a set of icons devoted to five types of visitors: the Citizen Scientist, the Developer, the Citizen Activist, the Govvie and the Curious. A great feature to engage NASA’s audience is the Data Stories section where people talk about the projects they created with NASA datasets.
The Data Briefing: Surrounded by Fields of Federal Data—U.S. Patent and Trademark Office’s PatentsView
Hello, everyone. My summer sabbatical was short but educational, and I am glad to be back in the federal government. I am also excited to again take up the weekly API article that is now expanded to include all things federal government data. Much has happened in the open data realm, and there is much to chronicle as government uses data in more innovative ways. On my sabbatical reading stack was “Code Halos: How the Digital Lives of People, Things, and Organizations are Changing the Rules of Business.
Take out your smartphone and count the number of apps that you have. How many of these apps do you use daily? What about the apps you use weekly? Do you have any apps that you installed but used only once? Any apps that you have never used? What kind of apps do you have? Are most of the apps used to communicate with friends and family? How many gaming apps do you have?
By now, you are familiar with “big data” or datasets that are so large that they cannot be analyzed by conventional analytical methods. You may have heard of “long data” which is data that has a temporal context. I work with long data when I analyze hiring patterns over time in workforce data. There is also “small data.” Small data are datasets that describe a current condition. For example, if you have a smart home appliance such as a smart thermostat or a home security system, that appliance is constantly monitoring data such as temperature or if a door is open.
Serendipity can be a wonderful tool for discovery. I was looking through the Census Bureau site for some business census data when I came upon the 2012 Census of Governments. According to the official description: “[t]he Census of Governments identifies the scope and nature of the nation’s state and local government sector; provides authoritative benchmark figures of public finance and public employment; classifies local government organizations, powers, and activities; and measures federal, state, and local fiscal relationships.
NASA has been busy since we last visited their collection of APIs back in August 2014. NASA has just launched API.NASA.gov where developers can learn to use existing NASA APIs or contribute their APIs to the catalog. NASA encourages developers to obtain an API Key to begin using or contributing APIs. Developers do not need an API key, but their requests to the API will be limited. I would encourage developers to obtain an API key.
The spring semesters are winding down at the universities where I teach. Many students are looking for summer internships or their first job after graduation. Of course, I talk about the opportunities in government through the Pathways program, the Presidential Management Fellows, or the various agency-specific internship programs. I’ve demonstrated USAJOBS in my classes, but I often wondered how to improve the experience for job seekers, especially for job seekers who prefer to use mobile apps.
Before coming to DC in late 2008, I lived in Louisville, Kentucky. Louisville is in the Ohio Valley Region, which meteorologists euphemistically call “weather-rich.” With spring came the beautiful flowers and the Kentucky Derby. Spring also brought flooding, tornadoes, severe thunderstorms, and windstorms. This is why I had several emergency weather radios that also doubled as flashlights and cell phone chargers. I also have several emergency information apps on my smartphone.
The Pew Research Center just released a report on how Americans view open government data. The following findings were based on a November to December 2014 survey of 3,212 adults. Two-thirds of Americans use the Internet or an app to connect with the government. According to Pew, 37% use the Internet to connect with the federal government, 34% connect with their state government, and 32% connect with their local government.
I recently found an app that provides a great service through crowdsourcing. Be My Eyes connects visually-impaired people with volunteers. Using the smartphone’s camera, the volunteers can perform tasks such as reading an expiration date or helping someone navigate unfamiliar surroundings. This is not a federal app, but I wanted to highlight it to demonstrate how crowdsourcing apps can make it easy for everyone to make a difference through microtasks.
I grew up when home computers were first being introduced to the general public. I bought my first computer, a Commodore 64, after spending a summer of mowing lawns and saving up my birthday and Christmas money. It was not until I entered college that I became an infopreneur. Infopreneurs are entrepreneurs who used computers and data sources to provide information products and services. My specialty was compiling information from the university’s collection of CD-ROMs that they received from various government agencies.
When I first started coding using BASIC on the Commodore 64, I rarely documented my programs. Neither did many of my fellow programmers which led to numerous hours trying to figure out just exactly how a program worked. Documentation became more vital as programs became more feature rich and complex. In the API world, there are a number of documentation standards to choose from when documenting an API.
The API Briefing: Free Federal Energy and Economic Information Delivered Straight to Your Spreadsheet
Back in November 2014, I wrote about the Federal Reserve of St. Louis’ FRED® (Federal Reserve Economic Data) API. A user can access 238,000 economic trends through FRED® through a website and mobile apps. What is unique about FRED® is that a user can pull economic data directly into an Excel spreadsheet. Now, the FRED® Excel plugin is joined by the U.S. Energy Information Administration’s (EIA) Excel plugin. The tool, which launched on March 18, incorporates both energy data from the EIA API and economic data from FRED®.
Instead of writing about a specific federal API this week, I want to talk about a new, evolving way of building Web interfaces and complete applications. Web Components allow developers to create their element that extends the HTML5 set of tags. Developers can create a Web Component that is a button that performs a specific function, such as composing and sending an email. Alternatively, a Web Component can be a complete application that a developer can easily drop into a Web page or mobile app.
The API Briefing: Fulfilling the D(e)SIRE for Renewable Energy with the Department of Energy’s New API
The Database of State Incentives for Renewables and Efficiency® (DSIRE®) provides information on incentives and policies for renewables and energy efficiency in the U.S. This joint project by the Department of Energy and North Carolina State University just released an API to query DSIRE®’s database. Developers can view the sample output by visiting the database query page. They can query by state or ZIP code to receive a listing of programs.
When browsing the various APIs offered by the federal government, you may have noticed that developers need to sign up for an API key. You may have also noticed that the documentation tells app developers to access the API using specified methods. Along with these two requirements, federal API creators have several ways to provide secure APIs for app developers and the general public. In this posting, I will describe how federal APIs are kept secure.
APIs and apps have been created for almost every aspect of human life. There are alarm clock apps, fitness apps, cooking apps, and personal finance apps, just to name a few of the thousands of apps available today. Most areas of society are well-represented in the app world except for one large portion of the American public—rural America. There need to be more apps for rural America. Fortunately, the U.
In my last posting, I argued that federal agencies should consider microservices architecture when releasing APIs. This is because allowing users to combine single-purpose apps together in unique ways helps people build personalized apps such as a driving map to local farmers markets. When given the opportunity, users will surprise you with the innovative creations they build from combining APIs. Just last week, the popular If This Then That (IFTTT) service released a federal-friendly Terms of Service.
DigitalGov recently spotlighted the National Highway Traffic Safety Administration’s (NHTSA) new SaferRide app. SaferRide provides safe alternatives to keep drunken drivers off the road. SaferRide uses the Yelp API to provide information about local taxi services for the part of the app where users can request a ride home. The SaferRide app is one example of how APIs can be mashed together to produce sophisticated applications. As APIs become more prevalent, there are two trends in app development that federal app developers should watch.
Big news in the technology world as Microsoft unveiled HoloLens and Microsoft’s use of holographic computing in the upcoming Windows 10 release. Holographic computing or augmented reality uses computer-generated images that are overlaid on real world videos. For example, a user can view a car through their smartphone. An app can project information such as make and model, fuel mileage, and other facts onto a real-time view of a particular car.
The federal government collects an amazing amount of economic data. Several agencies such as the Securities and Exchange Commission, the Treasury, and the Census Bureau collect economic data, ranging from the stock market activity to local business conditions. The Bureau of Labor Statistics (BLS) collects information on the labor market and is a rich source of data for researchers and the general public. The BLS offers two APIs for accessing labor data.
Recently, a reader pointed out that some of the APIs I write about are not really APIs but just datasets. Technically that is true but it only takes some development effort to turn a into an API. That is why I also highlight interesting federal datasets along with federal APIs. There are many federal datasets that should be APIs but how do agencies choose which datasets to build APIs?
According to some experts, over 80% of Americans will make a least one New Year’s resolution. There are the usual “lose weight,” “quit smoking,” or “exercise more” resolutions. Another popular set of resolutions involves learning new skills. So, if you are looking for a way to improve yourself while helping others, think about making a resolution to learn how to build a mobile app that can be used in disaster relief.
When websites were first created back in the 1990s, developers perfected their skills designing sites that presented content in an attractive and eye-catching manner. Content was completely contained within the four corners of the site. With the rise of Web 2.0, content creation became easier through blogs, wikis, and microblogging. Even so, content was still attached to that particular content creation tool. Content management systems (CMS) freed content from presentation.
We are in the middle of the holidays, and that means much driving to visit friends and relatives. I was just in Kentucky this past weekend where I spent a total of eight hours driving. I am sure many of you will spend even more time driving in the next three weeks. So, where do you find the best gas prices and how can you maximize your vehicle’s fuel mileage?
The Peace Corps just released a new dataset that lists all of the countries and regions Peace Corps volunteers serve. The API is RESTful and uses the JSON format. You have read in earlier columns about the different data formats for APIs and how to read the data presented by an API. As a refresher, I’ve created the following quiz based on the excellent documentation for the Peace Corps Countries and Regions API.
The federal government captures almost every economic data trend through several agencies. The Federal Reserve of St. Louis offers 238,000 economic trends through FRED® (Federal Reserve Economic Data). FRED® data can be accessed through the FRED® website or the FRED® mobile app (Android | Apple). FRED® data can even be pulled into Excel through a free plugin. Developers can take advantage of the vast data resources of FRED® and its cousin, ALFRED® (ArchivaL Federal Reserve Economic Data).
Every year, the Bureau of Justice Statistics surveys nearly 80,000 households and over 143,000 individuals about crime victimization. What is unique about this survey is that both reported and unreported crimes data is collected. The survey has a well-documented API which offers data in the CSV, XML, and JSON formats. Let’s examine the documentation to determine how a developer could use the data in the app. The National Crime Victimization Survey (NCVS) API is split into “personal victimization” data and “household victimization” data.
Back in 2000, I worked at a dot-com building website applications such as a real-time stock ticker ribbon and a real estate listings search engine. One of my favorite applications was an app for mobile phones. At that time, I used the Wireless Access Protocol (WAP), which displayed information using a special version of XHTML. Using the Kentucky Golf.com database, a user could search for information on a specific golf course by entering a ZIP code or using drop-down lists to search by county or course name.
Data.gov has 130,000+ datasets (as of November 3, 2014) many of which are designed for application developers. In previous columns, I’ve showcased some of the great applications built using federal APIs. Have you wondered where the idea for an app came from? Some developers start with an idea and then look for the API that best fits the idea. For example, a developer may want to create an app that alerts users of unsafe bus or limousine companies.
It is fall when the weather becomes colder, and people start firing up their furnaces. While I was working on putting in more insulation and installing a programmable thermostat, I wondered if the federal government has an API to help me lower my utility bills. Yes, and it is a great API! The Department of Energy (through the Lawrence Berkley National Laboratory) has the Home Energy Saver API which is a comprehensive service to analyze home energy use.
Access to clean water is fast becoming a vital issue in the 21st century. Changing climate patterns are drying up aquifers and limiting the amount of water runoff from thawing snow packs. Drought conditions in California are effecting hydroelectric production while dry conditions in the West have increased the frequency and harmful effects of forest fire. Monitoring and mapping water conditions across the U.S. is a vital government service.
The recent Ebola outbreaks demonstrate the need for current and authoritative health news. The Centers for Disease Control and Prevention (CDC) is the federal information source for Ebola and other infectious diseases, along with other public health data. Data.CDC.gov lists 48 datasets and views containing statistics from smoking to infectious diseases. Developers can use the Socrata Open Data API to pull JSON data into their apps. For those who are not developers, the CDC offers a way to embed health data into blogs, websites, and social media.
Glad to be back after a three-week absence. I was preparing for the South Eastern Conference on Public Administration held in Atlanta this year. Great conversations and I can tell you that the academic community is hungry for more government data and APIs. A great example is this week’s API Briefing: the U.S. National Library of Medicine’s TOXNET Web Service. TOXNET lists 16 databases ranging from TOXNET, which maps chemical releases, to the Hazard Substances Data Bank (HSDB).
If you have ever been a caregiver for an elderly family member or friend, you know that there are many resources to help you in giving care. But finding these resources can be difficult and frustrating. The Administration on Aging in the Department of Health and Human Services (HHS) has been guiding people to local resources since 1991. Starting with a phone service, the Administration on Aging created its first website in 2001.
The National Aeronautics and Space Administration (NASA) has an enormous collection of aerospace and science data sets. NASA missions and projects can create amazing amounts of data. One example: the Earth Observing System Data and Information System has collected enough information to fill the Library of Congress (Data.NASA.gov). A more recent example: the Solar Dynamics Observatory receives 1.5 terabytes of data a day. As NASA admits, this much information can be overwhelming for agency API development.
As the new school season approaches, it is a good time to see what federal datasets are available for educational app developers. Visit the developers’ page at ED.gov to find 36 educational datasets for educational levels. The datasets can be accessed in CSV, JSON, XML, and API formats. What is especially helpful is a PDF document that explains the data and the methodology behind the data collection. This is useful information for app developers when they combine datasets.
Up till now, all the APIs that have been written about in The API Briefing were read-only APIs. That means that information is only one way: from the API to the user or app. These APIs do allow limited interactivity in that the database behind the API can be searched, but the existing data cannot be edited, or new data added to the database. There are some federal government APIs that are writable.
As federal agencies release APIs on an almost daily basis, keeping track of the numerous datasets has become a vital service for developers. The Department for Health and Human Services (HHS) manages HealthData.Gov which currently lists 1,680 datasets in 36 categories such as “Public Health,” “Health Care Cost,” and “Health Statistics.” To help developers find relevant datasets and keep up with newly-added datasets, the HealthData.gov API was created. Developers can use the Catalog API to search the catalog and receive meta-information about a dataset in the JSON format.
Food deserts are areas where residents have little or no access to nutritional food. Food deserts exist because of low-incomes, lack of transportation, or too few stores that stock produce and other healthy food items. Governments from the local level to federal have implemented grant programs to encourage grocery store construction in the food deserts. Community activists have also worked to create food co-ops and encourage farmer markets to target the food deserts.
Once a federal agency releases an API, there are several ways they can be used in apps. The most common method is through hackathons. Hackathons are where an agency or agencies present the API(s) and invite developers to create prototype apps. The apps are then presented to subject matter experts for suggestions on creating the final app. There are many government hackathons on a variety of public issues. Visit Challenge.
It is easy to start a business today and especially an Internet-based business. Using the cloud, APIs, and hosted applications, an entrepreneur can quickly build a website/mobile app. The entrepreneur can hire freelancers to do everything from creating a logo to writing a business plan. Virtual assistant services can provide on-demand staff to meet business needs. Yes, it is easy to start a business. The hard part is creating and sustaining a business.
The API Briefing: How APIs Provide Localized Information – NOAA’s Weather Service Data and FCC’s Broadband Services Map
The two featured APIs this week are excellent demonstrations of personalizing federal government data by where a user lives. Federal agencies collect a considerable amount of community data, from the Census Bureau’s surveys to the FDA’s local agricultural conditions. Thanks to GPS, app developers can locate a user’s immediate geographical location and tailor information based on the latitude and longitude coordinates. The National Oceanic and Atmospheric Administration has an API that provides current conditions and a four-day forecast by locating the nearest NOAA station to the user’s geographical coordinates.
This week, we will look at three different APIs that demonstrate how agencies use different technologies to serve out data. Presenting data in various formats encourages developers to build on federal APIs. As past columns have shown, the innovative apps created with federal data are quickly growing. The latest API news this week is how quickly the Department of Labor (DOL) built a Software Developer Kit (SDK) for Apple’s new programming language.
The Census Bureau recently released a “machine-readable dataset discovery service” that lists 41 Census data sets. It’s in spreadsheet form and gives a description of the datasets along with links to the API and developer documentation. What makes the discovery service machine readable is that’s based on Project Open Data’s “Common Core Metadata Schema” that uses a standard way to describe and index government information sources. The discovery service makes it easier for developers to find and mix different APIs together to create sophisticated apps.
Not only does the Department of State have a great set of APIs, State also has an excellent example of how to build an informative and useful app. EducationUSA is a network of State Department advisers who help international students apply for U.S. university programs. The EducationUSA app has the most popular resources and services from the EducationUSA website, such as the ability to: Search for EducationUSA advising center information Follow the primary social media channels (Facebook, Twitter, & YouTube) View Frequently Asked Questions (in 8 languages) Discover new financial aid opportunities, and Utilize the Ask an Adviser (in five languages) function The EducationUSA app is an excellent example of designing for multiple-device experiences.
The Food and Drug Administration (FDA) just released the OpenFDA Research Project. At the heart of the project is the OpenFDA API, which allows developers to perform searches on FDA’s drug information database. Coming soon is the ability to search FDA information on medical devices and information about food. Visit the FDA’s API Basics page to learn how to access OpenFDA including interactive sample queries. The FDA’s API documentation is a great example of how to create detailed guidance for developers.
The Food and Drug Administration collects drug labeling information for human prescription, over-the-counter, homeopathic, and veterinary products through a special markup language called “Structured Product Labeling” (SPL). The database created from the SPL submissions is a treasure trove of health information that is valuable to pharmacists, doctors, and the ordinary health consumer. The problem is that data is hard for developers to access and process. Until recently, when the National Library of Medicine released open source code for “Pillbox.
Federal employee training is about to receive a much-needed boost in the President’s 2015 Budget Request. Training is essential to the federal workforce and agencies have a number of learning management systems to deliver online training along with the traditional classroom training. The problem is that all of these training sources don’t share information with each other about what training a learner has completed. Compiling a training record is a tedious and mostly manual process of printing out certificates, filling out SF-182s, and keeping paper records.
Around the D.C. area, one of the first signs of spring are the numerous farmers markets. In my neighborhood alone, I regularly visit four farmers markets that have a wide variety of produce and baked goods. Farmers markets are good for the local economy, and the easy access to fresh fruits and vegetables helps local communities. Realizing the importance of farmers markets, the USDA released the Farmers Market Directory API so that developers can create apps to help people find farmers markets in their area.
“PolicyOps” is a better way to create and implement government policies and programs through cutting-edge data analytics and new collaboration methods. PolicyOps (“Policy” plus “Operations”) is a new proposal for improving policy making and policy implementation. Based on a cutting-edge IT management method, DevOps (“Development” plus “Operations”), PolicyOps has two major concepts. First, closer collaboration and coordination between policy designers and policy implementers. Second, a single view of the policy environment reality agreed on by the policy designers and policy implementers.