Part 3: App Economy methodology and references: As noted in Part 1, we have developed a new, standardized methodology for estimating App Economy employment. This methodology can be applied to a wide variety of countries, languages, and economic environments. The methodology uses online job postings for workers with app-related skills as a real-time measure of App Economy employment. We benchmark this data against official government statistics in order to eliminate many of the well-known problems connected with using big data to measure economic variables.
Our new globally uniform methodology is built on a strong base of previous research, starting with the widely cited 2012 paper, “Where the Jobs Are: The App Economy” (see full list at end of document). For this study, a worker is in the App Economy if he or she is in:
How do we tell which jobs require App Economy skills? The key is to look at help wanted ads—also called job postings–where enterprises actually describe the skills and knowledge they are looking for. Our data source is the Indeed database of online job postings, found at Indeed.com. [i]
The heart of the analysis is the list of key words and phrases generally associated with App-Economy-related skills. In previous studies we have built up extended keyword lists. However, because we intend this analysis to be repeatable across a wide range of countries, we simplified the search terms.
The methodology consists of seven distinct steps.
1. Identification of App Economy job postings
Using summary statistics generated by searches on the Indeed website, we identify online job postings containing one of the following key words: “iOS” or “Android” or “Blackberry” or “Windows phone” or “Windows mobile” or “app”.
2. Validation
By the nature of the data, a keyword search for App Economy workers will typically include some inappropriate job postings. For example, the word ‘app’ can appear in a job posting for a truck driver who can use an app. Some postings for civilian jobs working for the Army suggest that the job-seeker download the “USAJOBS.gov app.” In some cases, an employment firm such as Kelly Services will use a keyword such as ‘app’ in its boilerplate for all job postings.
In order to adjust for these inappropriate job postings, we manually examine a sample of the job postings from step 1 to eliminate those that do not fit our criteria of an App Economy worker. This is a crucial part of the process. This allows us to estimate a validation ratio that we applied to the results of step 1.
3. Benchmarking IT job postings against official IT employment statistics
The Bureau of Labor Statistics issues an annual figure for the number of workers in all computer and mathematical occupations and computer and information system managers. We construct a keyword list to identify the corresponding job postings. This allows us to calculate the ratio of job postings to employment for overall IT occupations. [ii]
Benchmarking against official statistics is an essential step in any use of big data for economic analysis. It allows us to adjust for biases in the big data, both geographically and over time.
4. Estimation of App Economy core jobs for the United States
We assume that the ratio of online job postings to employment for overall IT occupations calculated in Step 3 also holds for core App Economy jobs. This is the key step in the estimation process.
We multiply the ratio generated in step 3 and the validated number of App Economy job postings generated in step 2. The result gives us the estimate of core App Economy jobs.
5. Estimation of total App economy employment for the United States
Using the same multipliers as in our previous work we estimated the total number of App Economy jobs in the United States. We assume that each core App Economy job is associated with two additional jobs (indirect and spillover jobs combined). This is a conservative assumption.
6. Estimation of the jobs that belong to the iOS, Android, Blackberry, or Windows Mobile/Phone ecosystems in the United States.
Out of the set of job postings containing the terms iOS, Android, Windows Mobile, Windows Phone, or Blackberry, we identify the share that contain terms belonging to the iOS ecosystem (Apple, iPad, iPhone, iOS); the share belonging to the Android ecosystem (Android, Google); the share belonging to the Blackberry ecosystem (Blackberry); and the share belonging to the Windows Mobile/Phone ecosystem (Windows Mobile, Windows Phone). Then those shares were applied to all App Economy employment. Note that these shares add up to more than 100 percent, because many job postings specify more than one mobile operating system (ie looking for an iOS/Android developer). Thus, a singlejob can belong to multiple ecosystems.
7. Estimation of App Economy employment in all 50 states and the District of Columbia
The Indeed database enables us to identify App Economy job postings by state. We then apply the same validation ratio and job posting to employment ratio found in steps 2 and 3.
Final Note
This methodology is an example of how big data produced by the private sector can be combined with existing government statistics to gain insight into a new and rapidly growing sector of the US economy. Moreover, because Indeed collects data about online job postings globally, the same methodology, with small adjustments, can be used to compare App Economy growth across the world.
Past App Economy Work
Broad Studies
June 2015: “A Low-Cost and Flexible Approach for Tracking Jobs and Economic Activity Related to Innovative Technologies; “ (Michael Mandel and Judith Scherer) South Mountain Economics/Nesta
May 2014: “Where Are the Big Data Jobs?” (Michael Mandel) Progressive Policy Institute
July 2013: “752,000 App Economy jobs on the 5th anniversary of the App Store” (Michael Mandel) Progressive Policy Institute.
October 2012:“The Geography of the App Economy” (Michael Mandel and Judith Scherer) South Mountain Economics/CTIA
February 2012: “Where the Jobs Are: The App Economy” (Michael Mandel) South Mountain Economics/Technet
Regional and Country Studies
September 2015: “Indonesia: Road to the App Economy” (Michael Mandel) Progressive Policy Institute
September 2015: “Vietnam and the App Economy” (Michael Mandel) Progressive Policy Institute
July 2014: “Jobs in the Australian App Economy” (Michael Mandel) Progressive Policy Institute
June 2014: “London: Digital City on the Rise” (Michael Mandel and Jonathan Liebenau) South Mountain Economics/Bloomberg Philanthropies
September 2013: “New York: Building A Digital City” (Michael Mandel) South Mountain Economics/Bloomberg Philanthropies
Notes
[i] Indeed calls itself “the world’s #1 job site, with over 180 million unique visitors every month.” Indeed is currently available for 56 countries, which helps make the globally-consistent methodology more straightforward.
[ii] Note that this ratio accounts for duplicate job postings, as well as job openings that are not publicly posted.
