This post was authored by Faisal, A'fif, Rosa and Fareed.
 |
CVE-2021-22205 is a critical remote code execution vulnerability in the service’s web interface. Initially, GitLab defined the issue at the time as an authenticated vulnerability produced by passing user-supplied images to the service's embedded version of ExifTool that was later assigned CVE-2021-22204 due to ExifTool's improper handling of DjVu files which a remote attacker could execute arbitrary commands as the git user.
At the moment, GitLab assigned this weakness CVE-2021-22205 and rated it a CVSSv3 score of 9.9. However, on September 21, 2021, GitLab amended the CVSSv3 score to 10.0. The rise in score was caused by shifting the vulnerability from an authenticated to an unauthenticated condition. Despite the minor difference in CVSS score, switching from authenticated to unauthenticated has significant repercussions for defenders. This vulnerability can be exploited remotely and does not require any kind of authentication or special expertise or access to the target area.
The NetbyteSEC team investigates one of the compromised Gitlab instances.
Analysis of the compromised
If a threat actor with network access to port 443 transmits an image with the DjVu annotation that contains malicious metadata, they can execute arbitrary commands on the server as the git user.
 |
| Figure 1: Gitlab access log |
 |
| Figure 2: Connection established to 194.145.227.21 |
Figure 2 shows the Gitlab server establishes connection to [194.145.227.21] - nbl37.ntup.net via port 5443. This IP address originated from Russia.
 |
| Figure 3: Process ID |
The NBS team initially investigates the process ID. Figure 3 shows the listing of all open files associated with the process ID of 145983.
 |
| Figure 4: Environment |
The team continues to check the environment of process 145983 as shown in the Figure 4, which resulting the process is started by git user in /tmp directory where the binary name is 0af5fc377c.
 |
| Figure 5: Maps table |
The NBS team continues by checking the maps table as in Figure 5 to show binary and libraries that the process is referring to when it runs. The maps table shows the binary that initiated the process, then it has been deleted. Threat actor has been dropped kthreaddk, which the binary has been deleted. The initial location:
/var/opt/gitlab/git-data/repositories/@hashed/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.wiki.git/objects/pack/0mvhrh/kthreaddk
 |
| Figure 6: sha1sum of the binary process |
The analyst checks for command line under cmdline and command name under comm for any info, both identical. Next, the team further by checking the binary hash as shown in Figure 6 and comparing it to VirusTotal for any related information.
SHA1: bd45c85a1b8e57890fc4dc30f416b9d65e30301e
Link to VirusTotal report:
https://www.virustotal.com/gui/file/67e38438759f34eaf50d8b38b6c8f18155bcc08a2e79066d9a367ea65e89aa3d/detection
From the VirusTotal report, this binary was reported as a miner. As we know, the miners will consume a high amount of system resources. For the verification purpose, the team uses the top command to monitor the CPU and memory usage.
|
| Figure 7: Resource usage |
Figure 7 shows the resource usage. The kthreaddk process uses higher memory and CPU usage than others, which is abnormal. This seems that the binary is likely a miner.
 |
| Figure 8: File descriptor |
Figure 8 shows the file descriptor where there is an open socket. This means another process might connect to this and be used. The process is likely to be created on Nov 12 at 1448 hrs (2:48pm). The analysis is continued by checking the inode number 1106832 with netstat command to get any info.
 |
| Figure 9: Open socket |
Figure 9 shows an open socket established. The inode number 1106832 is using a network socket as the same as the kthreaddk binary as shown in Figure 2. It is likely being used to transmit data or establish connection to threat actor crypto-miner pool's servers.
 |
| Figure 10: Deleted process binary |
Figure 10 shows the deleted process binary. As we know, the kthreaddk binary has been deleted before because the malware often self-deleted itself. However, we can recover the deleted process binary as long the process is still running. So, we avoided killing the suspected processes until we analyze or recover the binary first.
 |
| Figure 11: Recover deleted process binary |
Figure 11 shows the process of recovering deleted process binary. We can simply recover the deleted process binary by issuing a command as cp /proc/<PID>/exe /tmp/recover_binary_name. The NBS team copied the process binary to /tmp/recoverbin for further analysis.
Malware analysis
Netbytesec malware analysis team started the analysis by running the sample in our malware analysis lab box and the sample failed to be run due to the sample unable to find and open a configuration file named “config.json”. Figure below shows the error return after running the sample.
 |
| Figure 12: Error message |
Since the error of the sample shows “xmrig” string and looks like a behavior of XMRig application, the Netbytesec malware analysis team download the actual XMRig ELF application and make comparison between the sample and the xmrig application.
 |
| Figure 13: Comparison of XMrig and sample khtreaddk error message |
As a result of the comparison, the return error looks the same, except the original XMRig tries to find the configuration files at home directory and in the .config directory.
Due to the original config.json also being deleted from our Gitlab instance, the Netbytesec malware analysis team used a dummy sample of config.json file and put it in the same directory as the sample kthreaddk.
 |
| Figure 14: Running the malware sample with dummy config.json |
As a consequence, the sample is running successfully and the output is exactly the same as the XMRig application. As we can see, it’s XMRig version 6.4.0.
Observing the network connections in the lab shows that the cryptominer sample will be connected to the IP address that have been set in the configuration file.
 |
| Figure 15: Suspicious IP address established from the malware |
The address which belongs to the URL set in the configuration file stated in the “pools” section by the malware author.
 |
| Figure 16: URL in the config.json |
Nslookup of the domain displays the IP address which exactly matches the netstat result.
 |
| Figure 17: nslookup result |
To verify the miner sample is actually a XMRig application and look for other interesting malicious functions in the sample, the Netbytesec malware analysis team analyzed and reverse-engineered the inner code of the crypto-miner sample.
In the figure 18 below, the application will first load the address of the XMRig core function before being called by the function libc_start_main. Before that, a function at offset 0x7FA22AA5473C based on the figure below will be invoked to execute some syscall functions which are not malicious and intriguing like set pointer Thread ID, set architecture-specific thread state and enumerate environment variables.
 |
| Figure 18: main function |
In xmrig_core_functions, the Netbytesec malware analysis team tried to compare the code of the sample and XMRig ELF file. Both of them have the same pattern. The Netbytesec team has renamed a few of the functions in Ghidra for our code readability. Below figure 19 and 20 shows the code of the both XMRig application code and sample code.
 |
| Figure 19: Original XMRig ELF code |
 |
| Figure 20: Sample code |
Analyzing the core functions of XMRig in the malware code confirms that the code exactly matches the actual application XMRig that is being used to conduct the crypto miner behavior. The malware author likely reuse the code of XMRig of version 6.4.0 and embedded the code in the malware sample. The original source code of XMRig of version 6.4.0 can be found at XMRig GitHub repository [1].
OSINT research of the hash and network IOCs on the internet found the exact same hash and IP address in this Threat Intelligence repository by CUJO AI labs in GitHub [2].
Based on the resource [3], this malware is named as Sysrv Botnet. The malware core functionality is worm behavior and mine cryptocurrency coins. In the Sysrv botnet campaign, Sysrv continually incorporated new exploits to spread more effectively and is likely using CVE-2021-22205 as their new series of exploits as our Gitlab instance tagged as one of their random targets.
Indicator of Compromised (IOC)
Hash
6db4f74c02570917e087c06ce32a99f5
Bd45c85a1b8e57890fc4dc30f416b9d65e30301e - kthreaddk binary
Network
IP address: 194.145.227.21, Port:5443
Domain: nbl37.ntup.net
File dependency
Config.json
Indicator from Gitlab access log:
Method: POST
Endpoint: /uploads/user
Resources
- https://github.com/xmrig/xmrig/releases/tag/v6.4.0
- https://github.com/getCUJO/ThreatIntel/blob/master/IoC/sysrv_ioc
- https://cujo.com/the-sysrv-botnet-and-how-it-evolved/