Muse Security Evaluation

Update Notice

This website is outdated. Will automatically forward to the latest website in two seconds. If not, please click on this link to find the latest, updated website.

Discovering Flaws in Security-Focused Static Analysis Tools for Android using Systematic Mutation

Richard Bonett, Kaushal Kafle, Kevin Moran, Adwait Nadkarni, Denys Poshyvanyk

Overview

Mobile application security has been one of the major areas of security research in the last decade. Numerous application analysis tools have been proposed in response to malicious, curious, or vulnerable apps. However, existing tools, and specifically, static analysis tools, trade soundness of the analysis for precision and performance, and are hence soundy. Unfortunately, the specific unsound choices or flaws in the design of these tools are often not known or well-documented, leading to a misplaced confidence among researchers, developers, and users. This paper proposes the Mutation-based soundness evaluation (µSE) framework, which systematically evaluates Android static analysis tools to discover, document, and fix, flaws, by leveraging the well-founded practice of mutation analysis. We implement µSE as a semi-automated framework, and apply it to a set of prominent Android static analysis tools that detect private data leaks in apps. As the result of an in-depth analysis of one of the major tools, we discover 13 undocumented flaws. More importantly, we discover that all 13 flaws propagate to tools that inherit the flawed tool. We successfully fix one of the flaws in cooperation with the tool developers. Our results motivate the urgent need for systematic discovery and documentation of unsound choices in soundy tools, and demonstrate the opportunities in leveraging mutation testing in achieving this goal.

Discovered Security Flaws

Here we provide the list of all the security vulnerabilities that were discovered using μSE. Please, click on a vulnerability for more details.

Discovered Security Vulnerabilities

RunOnUIThread

Description:

Android Activities have access to the Activity.runOnUiThread() method, which accepts Runnables and executes their run() method on the UI thread. This example submits a Runnable containing a simple data leak within its run() method to the runOnUiThread() method.

Code Example:

    runOnUiThread(new Runnable() {
    @Override
    public void run() {
        String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
        android.util.Log.d("leak-1", dataLeak);
    }
});

ButtonOnClickToDialogOnClick

Description:

This snippet creates an AlertDialog with a listener that will execute onClick() when the user selects an item from the dialog’s list. The onClick() callback contains a data leak. This code is placed within a function registered as a Button’s callback in an XML layout file. Data is leaked when the user clicks the Button and then clicks an item in the AlertDialog’s list.

Code Example:

    public void leakData(View view) {
    CharSequence[] items = {"item1", "item2", "item3"};
    new AlertDialog.Builder(this).setTitle("example")
            .setCancelable(true)
            .setItems(items, new DialogInterface.OnClickListener() {
                @Override
                public void onClick(DialogInterface dialog, int i) {
                    String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
                    android.util.Log.d("leak-1", dataLeak);
                }
            })
            .create()
            .show();
}

Mutated Xml

    <Button
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:onClick="leakData" />

DialogFragmentShow

Description:

A DialogFragment contains a leak within its onCreateDialog() callback, which is called when the dialog is instantiated via DialogFragment.show().

Code Example:

   @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    DialogFragment example = new ExampleDialogFragment();
    example.show(getFragmentManager().beginTransaction(), "ExampleDialog");
}

public static class ExampleDialogFragment extends DialogFragment {
    @Override
    public Dialog onCreateDialog(Bundle savedInstanceState) {
        String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
        android.util.Log.d("leak-1", dataLeak);
        return new AlertDialog.Builder(getActivity()).setTitle("example")
                .create();
    }
}

NavigationView

Description:

An Activity implementing NavigationView.OnNavigationItemSelectedListener has an additional lifecycle callback onNavigationItemSelected() which is called when the user clicks on any item in the navigation menu. This example has a simple data leak within that lifecycle callback.

Code Example:

  public class MainActivity extends Activity implements NavigationView.OnNavigationItemSelectedListener {
    @Override
    public boolean onNavigationItemSelected(MenuItem item) {
        String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
        android.util.Log.d("leak-1", dataLeak);
        return false;
    }
}

ExecutorService

Description:

An ExecutorService manages threads, accepting Runnables and executing their run() methods in an available thread. In this sample, a simple data leak is contained within a Runnable passed to an ExecutorService that manages a single thread.

Code Example:

  ExecutorService service = Executors.newSingleThreadExecutor();
service.submit(new Runnable() {
    @Override
    public void run() {
        String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
        android.util.Log.d("leak-1", dataLeak);
    }
});

PhoneStateListener

Description:

A data leak is placed within a PhoneStateListener’s onDataConnectionStateChanged() callback, which is called when the device’s data connection state changes, such as when the device loses or regains connection to the internet.

Code Example:

  @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    TelephonyManager manager = (TelephonyManager) getSystemService(Context.TELEPHONY_SERVICE);
    manager.listen(new PhoneListener(), PhoneStateListener.LISTEN_CALL_STATE);
}

private class PhoneListener extends PhoneStateListener {
    @Override
    public void onDataConnectionStateChanged(int state) {
        String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
        android.util.Log.d("leak-1", dataLeak);
    }
}

BroadcastReceiver

Description:

Android BroadcastReceivers execute the onReceive() callback when an Intent matching the receiver’s IntentFilter is broadcast to the system. In this sample, a BroadcastReceiver dynamically registers another BroadcastReceiver in its onReceive() callback. The second receiver’s onReceive() callback contains a simple data leak.

Code Example:

  BroadcastReceiver receiver = new BroadcastReceiver() {
    @Override
    public void onReceive(Context context, Intent intent) {
        BroadcastReceiver receiver = new BroadcastReceiver() {
            @Override
            public void onReceive(Context context, Intent intent) {
                String dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
                android.util.Log.d("leak-1", dataLeak);
            }
        };
        IntentFilter filter = new IntentFilter();
        filter.addAction("android.intent.action.SEND");
        registerReceiver(receiver, filter);
    }
};
IntentFilter filter = new IntentFilter();
filter.addAction("android.intent.action.SEND");
registerReceiver(receiver, filter);

LocationListenerTaint

Description:

A LocationListener allows Android applications to react to changes in the user’s location. In this example, a LocationListener is registered to save a data source when the location provider’s status changes, such as when the user loses or regains cellular service, and leak the data when the user moves and changes location.

Code Example:

 @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    LocationManager locationManager = (LocationManager)getSystemService(LOCATION_SERVICE);
    locationManager.requestLocationUpdates(LocationManager.GPS_PROVIDER, 0, 0, locationListener);
}

private LocationListener locationListener = new LocationListener() {
    private String dataLeak = "";
    @Override
    public void onLocationChanged(Location location) {
        Log.d("leak", dataLeak);
    }

    @Override
    public void onStatusChanged(String provider, int status, Bundle extras) {
        dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
    }
};

NSDManager

Description:

An NsdManager.DiscoveryListener can be used to discover network services. In this example, a data leak source is placed within the onDiscoveryStarted() callback of the DiscoveryListener, which is called when the listener begins searching for services. Sinks are placed within the callbacks of a ResolveListener for either successful service resolution or failure, instantiated within the onServiceFound() callback of the DiscoveryListener.

Code Example:

 final NsdManager nsdManager = (NsdManager)this.getSystemService(Context.NSD_SERVICE);
NsdManager.DiscoveryListener listener = new NsdManager.DiscoveryListener() {
    String dataLeak = "";

    @Override
    public void onDiscoveryStarted(String serviceType) {
        dataLeak = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
    }

    @Override
    public void onServiceFound(NsdServiceInfo serviceInfo) {
        NsdManager.ResolveListener resolver = new NsdManager.ResolveListener() {

            @Override
            public void onResolveFailed(NsdServiceInfo serviceInfo, int errorCode) {
                Log.d("leak", dataLeak);
            }

            @Override
            public void onServiceResolved(NsdServiceInfo serviceInfo) {
                Log.d("leak", dataLeak);
            }
        };
        nsdManager.resolveService(serviceInfo, resolver);
    }
};
nsdManager.discoverServices("", NsdManager.PROTOCOL_DNS_SD, listener);

ListViewCallbackSequential

Description:

A ListView is instantiated with an onItemClickListener() whose onItemClick() callback will be called when an element in the ListView is selected. The callback first captures the element selected, casting it to the appropriate “Example” class, and then calls two functions within the class. The first function, “foo()” saves the source of a data leak to a private variable, and the second, “bar()”, leaks it.

Code Example:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    ArrayList<Example> examples = new ArrayList<Example>();
    examples.add(new Example());
    ListView serviceTable = (ListView) findViewById(R.id.listview);
    serviceTable.setAdapter(new ArrayAdapter<Example>(this,
            android.R.layout.simple_list_item_1, examples));
    serviceTable.setOnItemClickListener(new AdapterView.OnItemClickListener() {
        @Override
        public void onItemClick(AdapterView<?> parent, View view, int position, long id) {
            Example example = (Example) parent.getItemAtPosition(position);
            example.foo();
            example.bar();
        }
    });
}

private class Example {
    String dataLeAk;

    public void foo() {
        dataLeAk = Calendar.getInstance().getTimeZone().getDisplayName();
    }
    public void bar() {
        Log.d("leak", dataLeAk);
    }
}

ThreadTaint

Description:

A data leak source is saved to a variable within some method, and a Runnable containing a sink for the variable within its run() method is submitted to a Thread. It is important to note that the Thread is first saved to a variable before Thread.start() is called, as opposed to “new Thread(…).start()”.

Code Example:

final String dataLeak = Calendar.getInstance().getTimeZone().getDisplayName();
Thread thread = new Thread(new Runnable() {
    @Override
    public void run() {
        Log.d("leak-1", dataLeak);
    }
});
thread.start();

Fragments

Description:

A Fragment from the Android Support Library contains a simple data leak within its onCreateView() lifecycle callback. This callback is called when the Fragment is instantiated using the FragmentManager transaction mechanism.

Code Example:

public class LeakyFragment extends Fragment {
    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container,
                             Bundle savedInstanceState) {
        String dataLeak = Calendar.getInstance().getTimeZone().getDisplayName();
        Log.d("leak-1", dataLeak);
        return inflater.inflate(R.layout.fragment_leaky, container, false);
    }
}

public class MainActivity extends AppCompatActivity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        Fragment test = new LeakyFragment();
        getSupportFragmentManager().beginTransaction().replace(R.id.fragment, test).commit();
    }
}

SQLiteOpenHelper

Description:

A class extending the SQLiteOpenHelper abstract class contains a simple data leak within its onCreate() callback. This callback is called when the referenced database is created for the first time.

Code Example:

public class MySQLiteHelper extends SQLiteOpenHelper {
    public MySQLiteHelper(Context context) {
        super(context, "example", null, 1);
    }

    @Override
    public void onCreate(SQLiteDatabase db) {
        String dataLeak = Calendar.getInstance().getTimeZone().getDisplayName();
        Log.d("leak-1”, dataLeak);
    }
}

Mutation Operators & Schemes

Here we provide descriptions of the security mutation operator and mutation schemes implemented within μSE. Please, click on a mutation operator or scheme for more details.

Mutation Operator

DataLeakOperatorAST

Description:

Injects data leak variable declarations, sources, sinks, hops, and transformations at locations marked by a given mutation scheme. For example, the simple data leak used in our experiments used this replacement strategy:

Declaration: String dataLeak{{ IDENTIFIER }};
Source: dataLeak{{ IDENTIFIER }} = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
Sink: android.util.Log.d(“leak-{{ IDENTIFIER }}”, dataLeak{{ IDENTIFIER }});
Hop: dataLeak{{ IDENTIFIER }} = dataLeak{{ IDENTIFIER }};
Transformation: String temp{{ IDENTIFIER }} = dataLeak{{ IDENTIFIER }}; /* Transform temp based on a randomly selected rule that outputs an identical string */ dataLeak{{ IDENTIFIER }} = temp{{ IDENTIIFER }};

Detection Technique:

AST

Code Example:

Before (MIP from Complex-Path scheme with 1 hop)

class Example {
  // Declaration Mark 0
  // Declaration Mark 1
  // Declaration Mark 0-0
  // Declaration Mark 1-0

  void foo() {
    // Source Mark 0
    // Transformation Mark 1
    // Hop Mark 1-0
    // Transformation Mark 0-0
    // Sink Mark 0-0-0
    // Transformation Mark 1-0
    // Sink Mark 1-0-0
  }

  void bar() {
    // Source Mark 1
    // Transformation Mark 0
    // Hop Mark 0-0
    // Transformation Mark 0-0
    // Sink Mark 0-0-1
    // Transformation Mark 1-0
    // Sink Mark 1-0-1
  }
}

After

class Example {
  String dataLeak0;
  String dataLeak1;
  String dataLeak0_0;
  String dataLeak1_0;

  void foo() {
    dataLeak0 = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
    String temp1 = dataLeak1; Stringbuffer tempBuffer1 = new StringBuffer(); for (char char1 : temp1.toCharArray()) {tempBuffer1.append(char1);} temp1 = tempBuffer1.toString(); dataLeak1 = temp1;
    dataLeak1_0 = dataLeak1;
    String temp0_0 = dataLeak0_0; Stringbuffer tempBuffer0_0 = new StringBuffer(); for (char char0_0 : temp0_0.toCharArray()) {tempBuffer0_0.append(char0_0);} temp0_0 = tempBuffer0_0.toString(); dataLeak0_0 = temp0_0;
    android.util.Log.d(“leak-0-0-0”, dataLeak0_0);
    String temp1_0 = dataLeak1_0; Stringbuffer tempBuffer1_0 = new StringBuffer(); for (char char1_0 : temp1_0.toCharArray()) {tempBuffer1_0.append(char1_0);} temp1_0 = tempBuffer1_0.toString(); dataLeak1_0 = temp1_0;
    android.util.Log.d(“leak-1-0-0”, dataLeak1_0);
  }

  void bar() {
    dataLeak1 = java.util.Calendar.getInstance().getTimeZone().getDisplayName();
    String temp0 = dataLeak1; Stringbuffer tempBuffer0 = new StringBuffer(); for (char char0 : temp0.toCharArray()) {tempBuffer0.append(char0);} temp0 = tempBuffer0.toString(); dataLeak0 = temp0;
    dataLeak0_0 = dataLeak0;
    String temp0_0 = dataLeak0_0; Stringbuffer tempBuffer0_0 = new StringBuffer(); for (char char0_0 : temp0_0.toCharArray()) {tempBuffer0_0.append(char0_0);} temp0_0 = tempBuffer0_0.toString(); dataLeak0_0 = temp0_0;
    android.util.Log.d(“leak-0-0-1”, dataLeak0_0);
    String temp1_0 = dataLeak1_0; Stringbuffer tempBuffer1_0 = new StringBuffer(); for (char char1_0 : temp1_0.toCharArray()) {tempBuffer1_0.append(char1_0);} temp1_0 = tempBuffer1_0.toString(); dataLeak1_0 = temp1_0;
    android.util.Log.d(“leak-1-0-1”, dataLeak1_0);
  }
}

Mutation Schemes

ReachabilityAST

Description:

Marks every method with a declaration, source, and sink location consecutively.

Detection Technique:

AST

Code Example:

Before

void foo() {}

After

void foo() {
  // Declaration Mark 0
  // Source Mark 0
  // Sink Mark 0-0
}

AndroidAST

Description:

The Android scheme functions similarly to the reachability scheme, but only targets Android lifecycle methods, Android callbacks, intent messaging callbacks, and callbacks registered in XML layout files to reduce noise.

Detection Technique:

AST

Code Example:

Before

protected void onStop() {
  super.onStop();
}

After

protected void onStop() {
  super.onStop();
  // Declaration Mark 0
  // Source Mark 0
  // Sink Mark 0-0
}

Taint-BasedAST

Description:

For every method, first marks declarations at every parent class visible to the method, and corresponding sources for each declaration within the method. Then, at every method, marks sinks for every declaration mark visible to that method, aside from the declaration sourced by that method. Can be configured to send the variable through one or more hops before sinking the variable. When using hops, iteratively adds new variable declarations, passing the values of previously added declarations to the new ones. Only variables that have gone through the configured number of hops are sent to a sink.

Detection Technique:

AST

Code Example:

Before

class Example {
  void foo() {}
  void bar() {}
  class SubExample {
    void baz() {}
  }
}

After (0 hops)

class Example {
  // Declaration Mark 0
  // Declaration Mark 1
  // Declaration Mark 3
  void foo() {
    // Source Mark 0
    // Sink Mark 1-0
    // Sink Mark 3-0
  }

  void bar() {
    // Source Mark 1
    // Sink Mark 0-1
    // Sink Mark 3-1
  }

  class SubExample {
    // Declaration Mark 2
    void baz() {
      // Source Mark 2
      // Source Mark 3
      // Sink Mark 0-2
      // Sink Mark 1-2
      // Sink Mark 3-2
    }
  }
}

Complex-PathAST

Description:

Enhancement to the Taint-Based mutation scheme that adds a transformation mark before every sink and hop.

Detection Technique:

AST

Code Example:

Before

class Example {
  void foo() {}
  void bar() {}
  class SubExample {
    void baz() {}
  }
}

After

class Example {
  // Declaration Mark 0
  // Declaration Mark 1
  // Declaration Mark 3
  void foo() {
    // Source Mark 0
    // Transformation Mark 1
    // Sink Mark 1-0
    // Transformation Mark 3
    // Sink Mark 3-0
  }

  void bar() {
    // Source Mark 1
    // Transformation Mark 0
    // Sink Mark 0-1
    // Transformation Mark 3
    // Sink Mark 3-1
  }

  class SubExample {
    // Declaration Mark 2
    void baz() {
      // Source Mark 2
      // Source Mark 3
      // Transformation Mark 0
      // Sink Mark 0-2
      // Transformation Mark 1
      // Sink Mark 1-2
      // Transformation Mark 3
      // Sink Mark 3-2
    }
  }
}

Code & Data

Muse Code

Muse Minimal Apks & Data

Project Support & Funding

This material is based upon work supported by the National Science Foundation under Grant Number NSF-1815336