Should this handle packaged Chromium too?

[yoshimo]

Some applications run LIBCEF, aka Chromium Embedded Framework which in turn is using tls somewhere. Probably boringtls
I tried to attach fritrap on every spawned sub-process (It would be nice if fritap would automatically cover processes that are spawned from the main process as well), including those that seem to connect to the outside world.
The pcap stays empty. So i was wondering if this supposed to work or i am not doing anything wrong?

[monkeywave]

Hi @yoshimo

first of all thx for reporting this issue. I'm not sure if we tested friTap against Chrome explicitly and it might that friTap is currently not supporting LIBCEF based applications.
In order to improve friTap in that field can you suggest an example application we can use for testing and development purposes?
Further more on which platform do you encounter this issue?

[yoshimo]

Windows 10 Pro, https://download.battle.net/en-us/?product=bnetdesk

[milahu]

The pcap stays empty

same here, when attaching to a running chromium process

$ friTap -p fritap.pcap 113667

[*] Running Script on Linux
[*] libgnutls.so.30.37.0 found & will be hooked on Linux!
[*] libnspr4.so found & will be hooked on Linux!
[*] error: skipping module libnspr4.so
[*] Linux dynamic loader hooked.
[*] Logging TLS plaintext as pcap to fritap.pcap

also, friTap fails to spawn chromium

[*] Running Script on Linux
[*] libgnutls.so.30.37.0 found & will be hooked on Linux!
[*] libnspr4.so found & will be hooked on Linux!
[*] error: skipping module libnspr4.so
[*] Linux dynamic loader hooked.
[*] Logging TLS plaintext as pcap to fridatap.pcap
[*] libnspr4.so was loaded & will be hooked on Linux!
{'description': 'Could not find *libssl*.so!SSL_ImportFD', 'type': 'error'}
Terminated

friTap --debugoutput

note: the 113051 messages are from chromium

friTap \
  --spawn \
  --pcap $PWD/fridatap.pcap \
  --debugoutput \
  "$(which chromium) --user-data-dir=$PWD/chromium-user-data --disable-seccomp-sandbox --single-process https://httpbin.org/get"

Start logging
Press Ctrl+C to stop logging
spawning /nix/store/nsx8iznrwqwb4mwy3l9n4alvcd381z5k-ungoogled-chromium-unwrapped-120.0.6099.224/libexec/chromium/chromium --user-data-dir=/home/user/src/milahu/opensubtitles-scraper/aiohttp_firefox/chromium-user-data --disable-seccomp-sandbox --single-process https://httpbin.org/get
[113051:113051:0205/140145.787451:ERROR:system_network_context_manager.cc(854)] Cannot use V8 Proxy resolver in single process mode.
[113051:113051:0205/140145.792446:ERROR:policy_logger.cc(156)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(161) Cloud management controller initialization aborted as CBCM is not enabled. Please use the `--enable-chrome-browser-cloud-management` command line flag to enable it if you are not using the official Google Chrome build.
[113051:113051:0205/140146.175938:ERROR:system_network_context_manager.cc(854)] Cannot use V8 Proxy resolver in single process mode.
[*] capturing only plaintext data

(chromium:113051): dbind-WARNING **: 14:01:46.569: AT-SPI: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
[*] Running Script on Linux
[*] libgnutls.so.30.37.0 found & will be hooked on Linux!
[***] Found gnutls_record_recv 0x7ff3f1586a30
[***] Found gnutls_record_send 0x7ff3f15862c0
[***] Found gnutls_session_set_keylog_function 0x7ff3f159a2b0
[***] Found gnutls_transport_get_int 0x7ff3f1582860
[***] Found gnutls_session_get_id 0x7ff3f15a5240
[***] Found gnutls_init 0x7ff3f15be870
[***] Found gnutls_handshake 0x7ff3f1594fe0
[***] Found gnutls_session_get_keylog_function 0x7ff3f159a2a0
[***] Found gnutls_session_get_random 0x7ff3f15bfb70
[***] Found getpeername 0x7ff3f1a58c60
[***] Found getsockname 0x7ff3f1a58c90
[***] Found ntohs 0x7ff3f1a66e00
[***] Found ntohl 0x7ff3f1a66df0
[*] libnspr4.so found & will be hooked on Linux!
[***] Found PR_Write 0x7ff3f274eae0
[***] Found PR_Read 0x7ff3f274ead0
[***] Found PR_FileDesc2NativeHandle 0x7ff3f2769be0
[***] Found PR_GetPeerName 0x7ff3f274ec60
[***] Found PR_GetSockName 0x7ff3f274ec50
[***] Found PR_GetNameForIdentity 0x7ff3f274ffe0
[***] Found PR_GetDescType 0x7ff3f274eab0
[***] Found PK11_ExtractKeyValue 0x7ff3f28136f0
[***] Found PK11_GetKeyData 0x7ff3f28137a0
[*] error: skipping module libnspr4.so
[***] Loader error: Could not find *libssl*.so!SSL_ImportFD
[*] Linux dynamic loader hooked.
[*] Logging TLS plaintext as pcap to /home/user/src/milahu/opensubtitles-scraper/aiohttp_firefox/fridatap.pcap
[113051:113063:0205/140147.701222:ERROR:ev_root_ca_metadata.cc(162)] Failed to decode OID: 0
[*] libnspr4.so was loaded & will be hooked on Linux!
[***] Found PR_Write 0x7ff3f274eae0
[***] Found PR_Read 0x7ff3f274ead0
[***] Found PR_FileDesc2NativeHandle 0x7ff3f2769be0
[***] Found PR_GetPeerName 0x7ff3f274ec60
[***] Found PR_GetSockName 0x7ff3f274ec50
[***] Found PR_GetNameForIdentity 0x7ff3f274ffe0
[***] Found PR_GetDescType 0x7ff3f274eab0
[***] Found PK11_ExtractKeyValue 0x7ff3f28136f0
[***] Found PK11_GetKeyData 0x7ff3f28137a0
{'description': 'Could not find *libssl*.so!SSL_ImportFD', 'type': 'error'}
Terminated

running chromium in gdb

start chromium

chromium --user-data-dir=$PWD/chromium-user-data --disable-seccomp-sandbox --single-process

get the pid of the main process (other processes are renderer processes)

ps -AF | grep user-data-dir=$PWD/chromium-user-data | head -n1

attach

gdb -p 12345

add breakpoints

b read
b write

... or

b SSL_read
b SSL_write

navigate to some https website

chromium --user-data-dir=$PWD/chromium-user-data https://httpbin.org/get

watch out for the NetworkService thread

Thread 27 "NetworkService" hit Breakpoint 3.29, 0x00007fa8631e8e00 in read () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6
(gdb) where
#0  0x00007fa8631e8e00 in read () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6
#1  0x000055753a918851 in net::SocketPosix::ReadIfReady(net::IOBuffer*, int, base::OnceCallback<void (int)>) ()
#2  0x000055753a9165e9 in net::TCPSocketPosix::ReadIfReady(net::IOBuffer*, int, base::OnceCallback<void (int)>) ()
#3  0x000055753a8af79c in net::TCPClientSocket::ReadCommon(net::IOBuffer*, int, base::OnceCallback<void (int)>, bool) ()
#4  0x000055753a8a38dc in net::SocketBIOAdapter::BIOReadWrapper(bio_st*, char*, int) [clone .cfi] ()
#5  0x000055753a39d985 in BIO_read ()
#6  0x000055753a451e80 in bssl::ssl_handle_open_record(ssl_st*, bool*, bssl::ssl_open_record_t, unsigned long, unsigned char) ()
#7  0x000055753a436965 in ssl_read_impl(ssl_st*) [clone .llvm.2241782486724038753] ()
#8  0x000055753a4365c1 in SSL_read ()
#9  0x000055753a89f6ca in net::SSLClientSocketImpl::DoPayloadRead(net::IOBuffer*, int) ()
#10 0x000055753a89f595 in net::SSLClientSocketImpl::ReadIfReady(net::IOBuffer*, int, base::OnceCallback<void (int)>) ()
#11 0x000055753a8d50ea in net::SpdySession::PumpReadLoop(net::SpdySession::ReadState, int) ()
#12 0x000055753a8a00c1 in net::SSLClientSocketImpl::RetryAllOperations() ()
#13 0x000055753a8af86e in net::TCPClientSocket::DidCompleteRead(int) ()
#14 0x000055753a916686 in net::TCPSocketPosix::ReadIfReadyCompleted(base::OnceCallback<void (int)>, int) ()
#15 0x000055753a919094 in net::SocketPosix::OnFileCanReadWithoutBlocking(int) ()
#16 0x000055753a35fad2 in base::MessagePumpEpoll::WaitForEpollEvents(base::TimeDelta) ()
#17 0x000055753a35f332 in base::MessagePumpEpoll::Run(base::MessagePump::Delegate*) ()
#18 0x000055753a2fc11b in base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool, base::TimeDelta) ()
#19 0x000055753a2bac14 in base::RunLoop::Run(base::Location const&) ()
#20 0x000055753a31d5e8 in base::Thread::Run(base::RunLoop*) ()
#21 0x000055753a31d988 in base::Thread::ThreadMain() ()
#22 0x000055753a333133 in base::(anonymous namespace)::ThreadFunc(void*) [clone .a67435033112019129ad5c28ddc47327] [clone .cfi] ()
#23 0x00007fa863173333 in start_thread () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6
#24 0x00007fa8631f5efc in clone3 () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6

Thread 27 "NetworkService" hit Breakpoint 4, 0x000055cfe94aea84 in BIO_write ()
(gdb) where
#0  0x000055cfe94aea84 in BIO_write ()
#1  0x000055cfe9562fa2 in bssl::ssl_write_buffer_flush(ssl_st*) ()
#2  0x000055cfe9560b96 in bssl::do_tls_write(ssl_st*, unsigned long*, unsigned char, bssl::Span<unsigned char const>) ()
#3  0x000055cfe9560841 in bssl::tls_write_app_data(ssl_st*, bool*, unsigned long*, bssl::Span<unsigned char const>) [clone .cfi] ()
#4  0x000055cfe9547b18 in SSL_write ()
#5  0x000055cfe99b0b7b in net::SSLClientSocketImpl::DoPayloadWrite() ()
#6  0x000055cfe99b0a1c in net::SSLClientSocketImpl::Write(net::IOBuffer*, int, base::OnceCallback<void (int)>, net::NetworkTrafficAnnotationTag const&) ()
#7  0x000055cfe99e7a4c in net::SpdySession::PumpWriteLoop(net::SpdySession::WriteState, int) ()
#8  0x000055cfe93ede1c in base::TaskAnnotator::RunTaskImpl(base::PendingTask&) ()
#9  0x000055cfe940c43f in base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() ()
#10 0x000055cfe940cc95 in non-virtual thunk to base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() ()
#11 0x000055cfe947031e in base::MessagePumpEpoll::Run(base::MessagePump::Delegate*) ()
#12 0x000055cfe940d11b in base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool, base::TimeDelta) ()
#13 0x000055cfe93cbc14 in base::RunLoop::Run(base::Location const&) ()
#14 0x000055cfe942e5e8 in base::Thread::Run(base::RunLoop*) ()
#15 0x000055cfe942e988 in base::Thread::ThreadMain() ()
#16 0x000055cfe9444133 in base::(anonymous namespace)::ThreadFunc(void*) [clone .a67435033112019129ad5c28ddc47327] [clone .cfi] ()
#17 0x00007f04138e3333 in start_thread () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6
#18 0x00007f0413965efc in clone3 () from /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6

ssl functions in chromium

boringssl functions in chromium/src/third_party/boringssl/src/ssl/ssl_buffer.cc

• bssl::ssl_handle_open_record
• bssl::ssl_write_buffer_flush

boringssl functions in chromium/src/third_party/boringssl/src/ssl/s3_pkt.cc

• bssl::do_tls_write
• bssl::tls_write_app_data

boringssl functions in chromium/src/third_party/boringssl/src/crypto/bio/bio.c

• BIO_read
• BIO_write

chromium functions in chromium/src/net/socket/ssl_client_socket_impl.cc

• net::SSLClientSocketImpl::DoPayloadRead
• net::SSLClientSocketImpl::ReadIfReady
• net::SSLClientSocketImpl::RetryAllOperations
• net::SSLClientSocketImpl::DoPayloadWrite
• net::SSLClientSocketImpl::Write

CDP Network.dataReceived event in chromium

the Network.dataReceived event should be sent
when HTTP traffic is received... but the event is not sent

Network.streamResourceContent must be called
to enable the Network.dataReceived event, but no effect

Network.streamResourceContent

Enables streaming of the response for the given requestId. If enabled, the dataReceived event contains the data that was received during streaming.

https://source.chromium.org/chromium/chromium/src/+/main:out/Debug/gen/third_party/blink/renderer/core/inspector/protocol/network.cc

// ------------- Frontend notifications.

void Frontend::dataReceived(const String& requestId, double timestamp, int dataLength, int encodedDataLength, Maybe<Binary> data)
{
    if (!frontend_channel_)
        return;
    crdtp::ObjectSerializer serializer;
    serializer.AddField(crdtp::MakeSpan("requestId"), requestId);
    serializer.AddField(crdtp::MakeSpan("timestamp"), timestamp);
    serializer.AddField(crdtp::MakeSpan("dataLength"), dataLength);
    serializer.AddField(crdtp::MakeSpan("encodedDataLength"), encodedDataLength);
    serializer.AddField(crdtp::MakeSpan("data"), data);
    frontend_channel_->SendProtocolNotification(crdtp::CreateNotification("Network.dataReceived", serializer.Finish()));
}

https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/inspector/inspector_network_agent.cc

void InspectorNetworkAgent::DidReceiveData(uint64_t identifier,
                                           DocumentLoader* loader,
                                           const char* data,
                                           uint64_t data_length) {
  String request_id = RequestId(loader, identifier);
  Maybe<protocol::Binary> binary_data;

  if (data) {
    NetworkResourcesData::ResourceData const* resource_data =
        resources_data_->Data(request_id);
    if (resource_data && !resource_data->HasContent() &&
        (!resource_data->CachedResource() ||
         resource_data->CachedResource()->GetDataBufferingPolicy() ==
             kDoNotBufferData ||
         IsErrorStatusCode(resource_data->HttpStatusCode())))
      resources_data_->MaybeAddResourceData(request_id, data, data_length);

    if (streaming_request_ids_.Contains(request_id)) {
      binary_data =
          protocol::Binary::fromSpan(reinterpret_cast<const uint8_t*>(data),
                                     base::checked_cast<size_t>(data_length));
    }
  }

  GetFrontend()->dataReceived(
      request_id, base::TimeTicks::Now().since_origin().InSecondsF(),
      static_cast<int>(data_length),
      static_cast<int>(
          resources_data_->GetAndClearPendingEncodedDataLength(request_id)),
      std::move(binary_data));
}

i need this for my aiohttp_chromium to support capturing HTTP streams
because capturing streams is not supported by the Chrome DevTools Protocol (CDP)

one problem/challenge is that chromium has no dynamic linking to libssl.so or libboringssl.so

$ ldd $(which chromium) | grep ssl | wc -l
0

the naive attempt to hook BIO_read fails

Interceptor.attach(Module.getExportByName(null, 'BIO_read'), {
  onEnter(args) {
    // ...
  },
  onLeave(retval) {
    // ...
  },
});

Error: unable to find export 'BIO_read'

fix: use Interceptor.attach with function offsets

• https://learnfrida.info/basic_usage/#pointer-to-offsets
• https://stackoverflow.com/questions/68332781/frida-hook-native-non-exported-functions

$ nm $(which chromium) | grep -e BIO_read -e BIO_write

0000000007f42940 t BIO_read
0000000007f42a80 t BIO_write
0000000007f42b20 t BIO_write_all

... but this (hooking functions in the main executable)
seems to be impossible with frida
and instead, we need binary-patching tools like e9patch

see also E9Patch Web Browser Guide

It is also possible to instrument Google Chrome using E9Tool/E9Patch. However, for modern versions of Chrome, this can be troublesome:

• Chrome frequently uses data-in-code; and
• Chrome seems to copy some code to different locations at runtime. This breaks some of the basic assumptions for static binary rewriting.

considering that chromium is open source, this is ridiculous...
this obfuscation will be justified with "better security"
because so its harder to sniff HTTP traffic... but surely not impossible

see also frida/frida-tools#42

[monkeywave]

Hi,

thx for providing such detailed information. Currently, friTap can only identify SSL libraries when they are dynamically linked. However, if you know the offsets, you can try to specify them as explained in [1]. For this, use the --offsets parameter.

To identify newly spawned processes with friTap, you can leverage the spawn gating feature of Frida. Simply use the --enable_spawn_gating parameter to enable this functionality.

Regarding Chromium support, at present, other issues are prioritized due to the focus being broader than just a single application. However, we're always open to contributions. So, if you have a solution for this issue, please don't hesitate to share it with us :-)

[1] https://github.com/fkie-cad/friTap/blob/main/USAGE.md#providing-custom-offsetsaddresses

[milahu]

for the record, i dont need this for now, so im not working on this

one problem is that frida is slow, compared to gdb or lldb
the initial scanning of the binary is so much faster with gdb/lldb

see also kaliiiiiiiiii/Selenium-Driverless#123 (comment)

[monkeywave]

Hi,

this should be solved in the latest version of friTap (version 1.2.1.0).

In order to hook BoringSSL which is statically linked (without symbols) into Cronet we are able to extract the keys by hooking utilizing byte patterns.

Right now we developed that mainly for Android but soon we will extend the patterns for other platforms as well.

All the best

Daniel

[yoshimo]

Spawn gating does not yet work on Windows, the sample app as well as discord do not get hooked properly just yet.