May 15, 2019

Pentesting at Scale

Pentesting at Scale

We’ve been manually doing penetration testing on mobile apps at RIIS for over 8 years. Our oldest client has been regularly sending us apps since 2011 and we’ve gone through the OWASP top 10 mobile risks countless times trying to find vulnerabilities.We move our security engineers around a lot to keep the reports as fresh as possible. New eyes also see new things and bring new techniques that we’ve never looked at before.


Recently we were approached with a much longer list of apps to analyze. I’m not a huge fan of automating the security process as I think it’s one area where humans are much better than machines at finding exploits.However finding good security engineers isn’t easy. So we don’t seem to have any alternative as we can’t scale our staff to meet these new demands.You need the experience to know where to look and what tools to use. You need to be able to see where a small crack can be used to create a much larger breach and finally you need to be ridiculously stubborn to never let an app defeat you. That skill set is very hard to find. There’s also a timing issue as it takes a minimum of 5 days for each audit.So what happens when the number of tools increases and the number of apps you need to look at explodes? In this blog I’m going to talk about how we automated some of the analysis to speed up the process. We’re still using our engineers to look at the output of the tools but we’re automating how and when the tools are executed so there’s only a minimum of waiting around time.

Security Tools

We can break down the security tools that we want to use into the following categories. - -

  • Static Analysis (SAST)

  • Dynamic Analysis (DAST)

  • Open Source Intelligence (OSINT)

  • Offensive Scanning Tools (IAST)

In each category or bucket there can be anywhere from 6 to 12 tools that need to be run against the mobile app depending on whether it is an Android APK or an iOS IPA. Some tools run on both platforms, such as MobSF, some do not such as Qark (Android) and PassionFruit (iOS).

Building Jenkins Jobs

For the purposes of this blog we will be concentrating on MobSF and Qark to scan the mobile apps. We’ll use Jenkins to orchestrate running both tools in parallel against a collection of Android mobile applications.

Jenkins Setup

There are multiple ways of running Jenkins server. Jenkins works in a master/node architecture where the master is designed to provide the user interface and perform coordination and then the nodes do all the work. You also have the choice of how the software is installed and the way it runs. Both Jenkins master and nodes can run inside of a docker container, be directly installed on top of a host operating system or run in the cloud.

For our setup we will use a single machine acting both as Jenkins master and node running on a single but substantial hardware machine. We will also be leveraging multiple Jenkins plugins to help us along the way.

Setup Scanning Tools

Download and install the tools so they are available on the system and executable by the Jenkins user.

MobSF can be installed by pulling the docker container from docker hub using the command below. The restart=always option ensures that the container is always restarted even after host machine is rebooted

docker run -restart=always —it -p 8000:8000 opensecurity/mobile-security-framework-mobsf:latest

Qark is designed to look for several security related Android application vulnerabilities, either in source code or packaged APKs.  Install qark by running:

pip install qark

Qark is run on the command line and MobSF is called via a REST API.

Building the Jenkins Job

We’ll be using Jenkins pipelines in this setup, see Figure at the top of the page. We define each Jenkins job using the built in Jenkins Groovy domain specific language or DSL.Lets start off by creating an empty Jenkins pipeline job by navigating to Jenkins -> New Item. Assign a name to your job and select the type from the list to be ‘Pipeline’. Next lets start defining what we need the job to do.

First we need a way to pass in the filesystem location where we want the job to look for our mobile applications. Our basic pipeline script should look something like this after we add the parameters code. The parameters block will prompt us, for the path or ‘INPUT_LOCATION’, see Listing 1.

pipeline {  agent { any }  parameters {    string defaultValue: '', description: '', name: 'INPUT_LOCATION', trim: true  } }

Listing 1: Create your Pipeline

Next we want to get a list of the files we want the job to process. This can be accomplished by combining the dir() and findFiles() steps to give us an array of filenames, see Listing 2.

pipeline {  agent { any }  parameters {    string defaultValue: '', description: '', name: 'INPUT_LOCATION', trim: true    }    stage('Analysis') {    steps {      script {        dir(INPUT_LOCATION) {          files = findFiles(glob: '*.*')        }      }    }    } }

Listing 2: Find files to analyze

Listing 2 will return a list of files in our input directory. We’ll use this list as the input to our Jenkins job. The code to start our parallel task in Jenkins is in Listing 3.

pipeline {  agent { any }  parameters {    string defaultValue: '', description: '', name: 'INPUT_LOCATION', trim: true    }    stage('Analysis') {    steps {      script {        dir(INPUT_LOCATION) {          files = findFiles(glob: '*.*')        }               files.each { f ->          def TASK_COLLECTION = [:]          TASK_COLLECTION["MOBSF"] =  {          }          TASK_COLLECTION["Qark"] = {          }          parallel(TASK_COLLECTION)         }      }    }  } }

Listing 3: Creating Parallel MobSF and Qark tasks

Next we can flesh out how each individual task is going to be executed. For Qark all we need to do is execute the command and pass in the APK file, see Listing 4.

TASK_COLLECTION["Qark"] = {          sh "qark --apk ${env.INPUT_LOCATION}/${f}"}

Listing 4: Qark task

MobSF needs to make two API calls to get our analysis started. First we need to upload the file and collect the response and then secondly we need to initiate a scan. We will also need to pass our predefined MobSF API key, see Listing 5.

TASK_COLLECTION["MOBSF"] =  {    def AUTH_KEY = '1234567890'    upload_cmd = "curl -s -S -F 'file=@${env.INPUT_LOCATION}/${f}' http://localhost:8000/api/v1/upload -H 'Authorization:${AUTH_KEY}'"       upload_result = sh label: 'Upload Binary', returnStdout: true, script: upload_cmd    def response_map = readJSON text: upload_result    def app_type = response_map["scan_type"]       def app_hash = response_map["hash"]       def app_name = response_map["file_name"]    scan_start_cmd = "curl -s -S -X POST --url http://localhost:8000/api/v1/scan --data 'scan_type=${app_type}&file_name=${app_name}&hash=${app_hash}' -H 'Authorization:${AUTH_KEY}'"       sh label: 'Start Scan of Binary', returnStdout: true, script: scan_start_cmd }

Listing 5: MobSF task

Similarly we can add any number of other tools to our pipeline.

Running the Job

We start the Jenkins job, just like any other job. Figure 2 shows a completed APK analysis. We can see in the image that there’s a link to the output of the static analysis.

Figure 2: Jenkins Pipeline

Analyzing the Results

We ran the pipeline process against a RIIS Android app called ETA Detroit. There were lots of false positives that can safely be ignored. However within the data there was also some information that could not be overlooked. MobSF gave it a security score of 41/100 and pointed us at some SHA-1 Hashes that should be replaced. Qark identified a hardcoded API key that will need to be fixed.

Future Plans

We currently have an internal and an external version of the tool. The external tool doesn’t have every tool we’re using internally just yet but we’re working to add them in the next few weeks or months. The internal tool will also process many APKs at the same time. If you’d like to try the external tool then just click here. It will download any APK package name in Google Play and run it through our process, see Figure 3. To stop any abuse we automatically email the analysis results to the developer email listed on Google Play. Feel free to try it out on your own APK and let us know what you think.

Figure 3: Retrieve the APK from Google Play