Methodology
As noted in Part 1, we have developed a new, standardized methodology for estimating App Economy employment. This methodology can be applied to a wide variety of countries, languages, and economic environments. The methodology uses online job postings for workers with app-related skills as a real-time measure of App Economy employment. We benchmark this data against official government statistics in order to eliminate many of the well-known problems connected with using big data to measure economic variables.
Our new globally uniform methodology is built on a strong base of previous research, starting with the widely cited 2012 paper, “Where the Jobs Are: The App Economy” (see full list of previous studies at end of document). For this study, a worker is in the App Economy if he or she is in:
How do we tell which jobs require App Economy skills? The key is to look at help wanted ads—also called job postings–where enterprises actually describe the skills and knowledge they are looking for. Our data source is the Indeed job search site, which lists online job postings for each country. These various Indeed job search sites can be found at www.indeed.com/worldwide. [i]
Our goal is to estimate App Economy employment for the 28 members of the European Union, plus Norway and Switzerland. The job search site Indeed tracks job postings for 21 of these 30 countries. These are: Austria, Belgium, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Luxembourg, Netherlands, Norway, Poland, Portugal, Romania, Spain, Sweden, Switzerland, and United Kingdom. These 21 countries include roughly 95% of ICT professional job in the 30-country are.
The heart of the analysis is the list of key words and phrases generally associated with App-Economy-related skills. In previous studies we have built up extended keyword lists. However, because we intend this analysis to be repeatable across a wide range of countries, we simplified the search terms.
The methodology consists of seven distinct steps.
1. Identification of App Economy job postings
Using summary statistics generated by searches on the Indeed website, we identify online job postings containing one of the following key words: “iOS” or “Android” or “Blackberry” or “Windows Phone” or “Windows Mobile” or “app.”
2. Validation
By the nature of the data, a keyword search for App Economy workers will typically include some inappropriate job postings. For example, the word ‘app’ can appear in a job posting for a truck driver who needs to use an app on the job. The term ‘iOS’ can also refer to an island in Greece.
In order to adjust for these and other inappropriate job postings, we manually examine a sample of the job postings from step 1 to eliminate those that do not fit our criteria of an App Economy worker. This is a crucial part of the process. This allows us to estimate a validation ratio that we apply to the full results of step 1.
3. Benchmarking ICT job postings against official ICT employment statistics
Our methodology relies on benchmarking information and communications technology (ICT) job postings against official statistics. Benchmarking against official statistics is an essential step in any use of big data for economic analysis. It allows us to adjust for biases in the underlying job posting data, both geographically and over time.
For each country, we construct a keyword list to identify ICT job postings in that country. We start with a common base of search terms in English, and then for each country, add a set of corresponding search terms in that country’s main language or languages (in the case of countries such as Belgium, Luxembourg, and Switzerland). For example, the search terms for Germany ICT job postings includes such terms as “web-entwickler” and “netzwerkadministrator.”
For our European analysis, we benchmark the job postings against figures on the number of ICT professionals per country, drawn from the International Labor Organization (ILO) database, which in turn gets its numbers from national surveys. We then use a conservative assumption relating the number of ICT professionals to the size of the whole ICT workforce in the country.[ii] After validation, this allows us to calculate the ratio of job postings to employment for overall ICT occupations for each country.[iii]
4. Estimation of App Economy core jobs for European countries
We assume that the ratio of online job postings to employment for overall ICT occupations calculated in Step 3 also holds for core App Economy jobs. This is the key step in the estimation process.
We multiply the ratio generated in step 3 and the validated number of App Economy job postings generated in step 2. The result gives us the estimate of core App Economy jobs.
5. Estimation of total App economy employment for European countries
Using the same multipliers as in our previous work we estimated the total number of App Economy jobs in each European country. We assume that each core App Economy job is associated with two additional jobs (indirect and spillover jobs combined). Once again, this is a conservative assumption compared to other studies.
6. Estimation of the jobs that belong to the iOS, Android, Blackberry, or Windows Mobile/Phone ecosystems in European countries
Out of the set of job postings containing the terms iOS, Android, Windows Mobile, Windows Phone, or Blackberry, we identify the share that contain terms belonging to the iOS ecosystem (Apple, iPad, iPhone, iOS); the share belonging to the Android ecosystem (Android, Google); the share belonging to the Blackberry ecosystem (Blackberry); and the share belonging to the Windows Mobile/Phone ecosystem (“Windows Mobile”, “Windows Phone”). Then those shares were applied to all App Economy employment. Note that these shares add up to more than 100 percent, because many job postings specify more than one mobile operating system (i.e. looking for an iOS/Android developer). Thus, a single job can belong to multiple ecosystems.
7. Estimating App Economy jobs for EU-28 plus 2.
This methodology allows us to estimate App Economy jobs for the 21 countries covered by Indeed. There are nine countries in our target set which Indeed does not cover. These are Bulgaria, Croatia, Cyprus, Estonia, Latvia, Lithuania, Malta, Slovakia, and Slovenia. In total they account for only a small share of the total target population, so we account for their effect by assuming that they have the same app intensity as the average for the other 21 countries.
Final Note
This methodology is an example of how big data produced by the private sector can be combined with existing government statistics to gain insight into a new and rapidly growing sector of the global economy. Moreover, because Indeed collects data about online job postings globally, the same methodology, with small adjustments, can be used to compare App Economy employment in countries and regions across the world.
Past App Economy Work
Broad Studies
June 2015: “A Low-Cost and Flexible Approach for Tracking Jobs and Economic Activity Related to Innovative Technologies,“ (Michael Mandel and Judith Scherer) South Mountain Economics/Nesta
May 2014: “Where Are the Big Data Jobs?” (Michael Mandel) Progressive Policy Institute
July 2013: “752,000 App Economy jobs on the 5th anniversary of the App Store” (Michael Mandel) Progressive Policy Institute.
October 2012:“The Geography of the App Economy” (Michael Mandel and Judith Scherer) South Mountain Economics/CTIA
February 2012: “Where the Jobs Are: The App Economy” (Michael Mandel) South Mountain Economics/Technet
Regional and Country Studies
January 2016: App Economy jobs in the United States (Part 1) (Michael Mandel) Progressive Policy Institute
September 2015: “Indonesia: Road to the App Economy” (Michael Mandel) Progressive Policy Institute
September 2015: “Vietnam and the App Economy” (Michael Mandel) Progressive Policy Institute
July 2014: “Jobs in the Australian App Economy” (Michael Mandel) Progressive Policy Institute
June 2014: “London: Digital City on the Rise” (Michael Mandel and Jonathan Liebenau) South Mountain Economics/Bloomberg Philanthropies
September 2013: “New York: Building A Digital City” (Michael Mandel) South Mountain Economics/Bloomberg Philanthropies
Notes
[i] Indeed calls itself “the world’s #1 job site, with over 180 million unique visitors every month.” Indeed is currently available for 56 countries, which helps make the globally-consistent methodology more straightforward.
[ii] ICT professionals correspond to ISCO-08 code 25. For the US, we benchmarked job postings to the sum of computer and information systems managers (SOC code 11-3020) and computer and mathematical occupations (SOC 15).
[iii] Note that this ratio accounts for duplicate job postings, as well as job openings that are not publicly posted.
