Merge pull request #2 from erev0s/more_info

report evasion techniques discovered

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.0.4] - 11-10-2023
+ - Added functionality to report back which static analysis evasion techniques were used
+ - New flag in the CLI to use the added functionality of the library
+
 ## [1.0.3] - 08-10-2023
  - New method get_manifest() in manifestDecoder module for convenience when getting the decoded AndroidManifest from an apk or a file.
  - added flag in the cli to be able to pass an encoded AndroidManifest.xml file and decode it.

README.md

@@ -3,14 +3,29 @@
 # apkInspector
 apkInspector is a tool designed to provide detailed insights into the central directory and local headers of APK files, offering the capability to extract content and decode the AndroidManifest.xml file. What sets APKInspector apart is its adherence to the zip specification during APK parsing, eliminating the need for reliance on external libraries. This independence, allows APKInspector to be highly adaptable, effectively emulating Android's installation process for APKs that cannot be parsed using standard libraries. The main goal is to enable users to conduct static analysis on APKs that employ evasion techniques, especially when conventional methods prove ineffective.
 
+## How to install
+apkInspector is available through PyPI
+~~~~
+pip install apkInspector
+~~~~
+
+or you can clone this repository and build and install locally:
+~~~~
+git clone https://github.com/erev0s/apkInspector.git
+cd apkInspector
+poetry build
+pip install dist/apkInspector-Version_here.tar.gz
+~~~~
+
 ## CLI
 apkInspector offers a command line tool with the same name, with the following options;
 
 ~~~~
 $ apkInspector -h
-usage: apkInspector [-h] [-apk APK] [-f FILENAME] [-ll] [-lc] [-la] [-e] [-x] [-xa] [-m] [-v]
+usage: apkInspector [-h] [-apk APK] [-f FILENAME] [-ll] [-lc] [-la] [-e] [-x] [-xa] [-m] [-sm SPECIFY_MANIFEST] [-a] [-v]
 
-APK Inspector
+apkInspector is a tool designed to provide detailed insights into the central directory and local headers of APK files, offering the capability to extract
+content and decode the AndroidManifest.xml file.
 
 options:
   -h, --help            show this help message and exit
@@ -24,28 +39,43 @@ options:
   -x, --extract         Attempt to extract the file specified by the -f flag
   -xa, --extract-all    Attempt to extract all files detected in the central directory header
   -m, --manifest        Extract and decode the AndroidManifest.xml
+  -sm SPECIFY_MANIFEST, --specify-manifest SPECIFY_MANIFEST
+                        Pass an encoded AndroidManifest.xml file to be decoded
+  -a, --analyze         Check an APK for static analysis evasion techniques
   -v, --version         Retrieves version information
 ~~~~
 
 
 ## Library
 The library component of apkInspector is designed with extensibility in mind, allowing other tools to seamlessly integrate its functionality. This flexibility empowers developers to leverage the capabilities of apkInspector within their own applications and workflows. To facilitate clear comprehension and ease of use, comprehensive docstrings accompany all primary methods, providing valuable insights into their functionality, expected arguments, and return values. These detailed explanations serve as invaluable guides, ensuring that developers can quickly grasp the inner workings of apkInspector's core features and smoothly incorporate them into their projects.
 
+### Features offered
+ - Find end of central directory record
+ - Parse central directory of APK and get details about each entry
+ - Get details local header for each entry
+ - Extract single or all files within an APK
+ - Decode AndroidManifest.xml file
+ - Identify Tampering Indicators:
+   - End of Central Directory record defined multiple times
+   - Unknown compression methods
+   - Unexpected starting signature of AndroidManifest.xml
+   - Tampered StringCount value
+   - Dummy data between elements
+
+
 The command-line interface (CLI) serves as a practical illustration of how the methods provided by the library have been employed.
 
 ## Planned todo
- - Return indicators of what was detected to be tampered.
+ - Proper documentation
  - Improve code coverage
 
 ## Disclaimer
 It should be kept in mind that apkInspector is an evolving project, a work in progress. As such, users should anticipate occasional bugs and anticipate updates and upgrades as the tool continues to mature and enhance its functionality. Your feedback and contributions to apkInspector are highly appreciated as we work together to improve and refine its capabilities.
 
 ## How to Contribute
-
 We welcome contributions from the open-source community to help improve and enhance apkInspector. Whether you're a developer, tester, or documentation enthusiast, your contributions are valuable. Here's how you can get started:
 
 ### Reporting Issues
-
 If you encounter a bug, have a feature request, or have a suggestion for improvement, please open an issue on our [GitHub issue tracker](https://github.com/erev0s/apkInspector/issues). When reporting issues, please provide as much detail as possible, including:
 
 - A clear and descriptive title

apkInspector/__init__.py

@@ -1 +1 @@
-__version__ = "1.0.3"
+__version__ = "1.0.4"

apkInspector/extract.py

@@ -25,21 +25,24 @@ def extract_file_based_on_header_info(apk_file, offset, header_info):
     if compression_method == 0:  # Stored (no compression)
         compressed_data = apk_file.read(compressed_size)
         extracted_data = compressed_data
+        indicator = 'STORED'
     elif compression_method == 8:
         compressed_data = apk_file.read(compressed_size)
         # -15 for windows size due to raw stream with no header or trailer
         extracted_data = zlib.decompress(compressed_data, -15)
+        indicator = 'DEFLATED'
     else:
-        # Any ZIP compression method other than STORED is treated as DEFLATED by Android.
         try:
             cur_loc = apk_file.tell()
             compressed_data = apk_file.read(compressed_size)
             extracted_data = zlib.decompress(compressed_data, -15)
+            indicator = 'DEFLATED_TAMPERED'
         except:
             apk_file.seek(cur_loc)
             compressed_data = apk_file.read(uncompressed_size)
             extracted_data = compressed_data
-    return extracted_data
+            indicator = 'STORED_TAMPERED'
+    return extracted_data, indicator
 
 
 def extract_all_files_from_central_directory(apk_file, central_directory_entries, output_dir):
@@ -64,7 +67,7 @@ def extract_all_files_from_central_directory(apk_file, central_directory_entries
             # Retrieve the header information for the file
             _, header_info = headers_of_filename(apk_file, central_directory_entries, filename)
             # Extract the file using the local header information
-            extracted_data = extract_file_based_on_header_info(apk_file, local_header_offset, header_info)
+            extracted_data = extract_file_based_on_header_info(apk_file, local_header_offset, header_info)[0]
             # Construct the output file path
             output_path = os.path.join(output_dir, filename)
             # Create directories if necessary

apkInspector/headers.py

@@ -176,36 +176,38 @@ def print_headers_of_filename(cd_h_of_file, local_header_of_file):
         print(f"{k:40} : {local_header_of_file[k]}")
 
 
-def show_and_save_info_of_central(central_directory_entries, apk_name, export: bool):
+def show_and_save_info_of_central(central_directory_entries, apk_name, export: bool, show: bool):
     """
     Print information for each entry for the central directory header and allow to possibly export to JSON
     :param central_directory_entries: The dictionary with all the entries for the central directory (see parse_central_directory)
     :param apk_name: String with the name of the APK, so it can be used for the export.
     :param export: Boolean for exporting or not to JSON
+    :param show: Boolean for printing or not the entries
     """
-    if not export:
+    if show:
         for entry in central_directory_entries:
             pretty_print_header(entry)
             print(central_directory_entries[entry])
-    else:
+    if export:
         save_to_json(f"{apk_name}_central_directory_header.json", central_directory_entries)
 
 
-def get_and_save_local_headers_of_all(apk_file, central_directory_entries, apk_name, export: bool):
+def get_and_save_local_headers_of_all(apk_file, central_directory_entries, apk_name=None, export: bool = None, show: bool = None):
     """
     Creates a dictionary of local headers based on the entries retrieved from the central directory header.
     Additionally, allows to print the local headers and export the dictionary to JSON
     :param apk_file: The already read/loaded data of the APK file e.g. with open('test.apk', 'rb') as apk_file
     :param central_directory_entries: The dictionary with all the entries for the central directory (see parse_central_directory)
     :param apk_name: String with the name of the APK, so it can be used for the export.
     :param export: Boolean for exporting or not to JSON. If exporting it will not print.
+    :param show: Boolean for printing or not the entries
     :return: Returns the dictionary created with all the local headers, where the filename is the key.
     """
     local_headers = {}
     for entry in central_directory_entries:
         entry_local_header = parse_local_header(apk_file, central_directory_entries[entry])
         local_headers[entry_local_header['Filename']] = entry_local_header
-        if not export:
+        if show:
             pretty_print_header(entry_local_header['Filename'])
             print(entry_local_header)
     if export:

apkInspector/indicators.py

@@ -0,0 +1,85 @@
+import io
+
+from .extract import extract_file_based_on_header_info
+from .headers import find_eocd, parse_central_directory, get_and_save_local_headers_of_all, headers_of_filename
+from .manifestDecoder import ResChunkHeader, StringPoolType, process_headers
+
+
+def count_eocd(apk_file):
+    """
+
+    :param apk_file:
+    :return:
+    """
+    content = apk_file.read()
+    return content.count(b'\x50\x4b\x05\x06')
+
+
+def zip_tampering_indicators(apk_file):
+    """
+
+    :param apk_file:
+    :return:
+    """
+    zip_tampering_indicators_dict = {}
+    count = count_eocd(apk_file)
+    if count > 1:
+        zip_tampering_indicators_dict['eocd_count'] = count
+    eocd = find_eocd(apk_file)
+    central_directory_entries = parse_central_directory(apk_file, eocd["Offset of start of central directory"])
+    local_headers = get_and_save_local_headers_of_all(apk_file, central_directory_entries)
+
+    for key in central_directory_entries:
+        cd_entry = central_directory_entries[key]
+        lh_entry = local_headers[key]
+        temp = {}
+        if cd_entry['Compression method'] not in [0, 8]:
+            temp['central compression method'] = cd_entry['Compression method']
+        if lh_entry['Compression method'] not in [0, 8]:
+            temp['local compression method'] = lh_entry['Compression method']
+        if cd_entry['Compression method'] not in [0, 8] or lh_entry['Compression method'] not in [0, 8]:
+            indicator = \
+                extract_file_based_on_header_info(apk_file, cd_entry["Relative offset of local file header"], lh_entry)[
+                    1]
+            temp['actual compression method'] = indicator
+        if not temp:
+            continue
+        zip_tampering_indicators_dict[key] = temp
+    return zip_tampering_indicators_dict
+
+
+def manifest_tampering_indicators(manifest):
+    """
+
+    :param manifest:
+    :return:
+    """
+    chunkHeader = ResChunkHeader.from_file(manifest)
+    manifest_tampering_indicators_dict = {}
+    if chunkHeader.type != 3:
+        manifest_tampering_indicators_dict['file_type'] = chunkHeader.type
+    string_pool = StringPoolType.from_file(manifest)
+    if len(string_pool.string_offsets) != string_pool.header.string_count:
+        manifest_tampering_indicators_dict['string_pool'] = {'string count': string_pool.header.string_count,
+                                                             'real string count': len(string_pool.string_offsets)}
+    dummy = process_headers(manifest)[1]
+    if dummy:
+        manifest_tampering_indicators_dict['dummy data'] = 'found'
+    return manifest_tampering_indicators_dict
+
+
+def apk_tampering_check(apk_file):
+    """
+
+    :param apk_file:
+    :return:
+    """
+    zip_tampering_indicators_dict = zip_tampering_indicators(apk_file)
+    eocd = find_eocd(apk_file)
+    central_directory_entries = parse_central_directory(apk_file, eocd["Offset of start of central directory"])
+    cd_h_of_file, local_header_of_file = headers_of_filename(apk_file, central_directory_entries,
+                                                             "AndroidManifest.xml")
+    offset = cd_h_of_file["Relative offset of local file header"]
+    manifest = io.BytesIO(extract_file_based_on_header_info(apk_file, offset, local_header_of_file)[0])
+    manifest_tampering_indicators_dict = manifest_tampering_indicators(manifest)
+    return {'zip tampering': zip_tampering_indicators_dict, 'manifest tampering': manifest_tampering_indicators_dict}

apkInspector/manifestDecoder.py

@@ -347,9 +347,10 @@ def process_headers(file):
     Takes into account that the resource map, the start namespace and end namespace chunks are only to be found once
     within the file.
     :param file: the xml file after the string pool chunk
-    :return: Returns all the elements found as their corresponding classes
+    :return: Returns all the elements found as their corresponding classes and whether dummy data were found in between
     """
     elements = []
+    dummy = 0
     possible_headers = {b'\x80\x01', b'\x00\x01', b'\x02\x01', b'\x03\x01', b'\x01\x01'}
     while True:
         # Parse the next header
@@ -361,12 +362,13 @@ def process_headers(file):
             break
         if check not in possible_headers:
             file.read(1)
+            dummy = 1
             continue
         chunk_type = parse_next_header(file)
         elements.append(chunk_type)
         if check in {b'\x80\x01', b'\x00\x01', b'\x01\x01'}:
             possible_headers.remove(check)
-    return elements
+    return elements, dummy
 
 
 def create_manifest(elements, string_data):
@@ -430,7 +432,7 @@ def get_manifest(file_like_object):
     ResChunkHeader.from_file(file_like_object)
     string_pool = StringPoolType.from_file(file_like_object)
     string_data = string_pool.strdata
-    elements = process_headers(file_like_object)
+    elements = process_headers(file_like_object)[0]
     manifest = create_manifest(elements, string_data)
     return manifest