Shut the Door: Guarding Against SonicWall GMS Remote Code Execution (CVE-2021-20020)

The Advanced Research Team at CrowdStrike Intelligence discovered two vulnerabilities in SonicWall Global Management System 9.3 (GMS) that, when combined, allow unauthenticated attackers to remotely execute arbitrary code with root privileges. SonicWall issued a patch for the vulnerabilities in question in April 2021 and immediately notified impacted customers and partners. For more information, please refer to SonicWall PSIRT Advisory ID SNWLID-2021-0009.

In this blog post, we outline the technical details and walk you through the steps from discovery to exploitation. And, we explain which indicators of compromise to look for in forensic evidence and how to prevent the attack.

Overview

Product Global Management System (GMS) 9.3
Affected Versions (without claim for completeness) 9.3.9314
Fixed Version GMS 9.3 MAR-22474.1-HotFix (2021-04-07)
CVE CVE-2021-20020
Root Cause Password-less PostgreSQL service on port 5029/tcp (“Trust Authentication”) and world-writable/etc. directory
Impact Unauthenticated remote code execution as postgres and local privilege escalation to root
SonicWall Resources Advisory: “SonicWall GMS 9.3 Unauthenticated Remote Command Execution Vulnerability”

Technical Details

As a starting point for our research, we set up a SonicWall GMS appliance in its default configuration. Once the initial setup was completed and the regular web interface became reachable, the remote attack surface was analyzed by conducting a port scan. It was found that the appliance exposes a PostgreSQL service on TCP port 5029. Further analysis showed that the database operates in “Trust Authentication” mode for all database users connecting from arbitrary IPv4 addresses. This effectively allows anyone to authenticate as an arbitrary database user, including the superuser postgres, without providing a password.

On the appliance, the corresponding configuration line can be found in the file pg_hba.conf:

[root@gms ~]# tail -n 14 /var/lib/infobright_pg/pg_data/pg_hba.conf 
# TYPE  DATABASE        USER            ADDRESS                 METHOD
[...]
host     all            all             0.0.0.0/0               trust

Using the psql tool, we can easily confirm that remote superuser access is indeed possible without any authentication:

$ psql -h 192.168.135.131 -p 5029 -U postgres -c "SELECT version()"
                                                             version                                                        
      
----------------------------------------------------------------------------------------------------------------------------
------
 PostgreSQL 9.2.2 (IB_33928), shared on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-44), 6
4-bit
(1 row)

The command initiates a connection to the database (at 192.168.135.131) as the superuser postgres and invokes the built-in version() function, which returns the database’s version information as a result. This step should normally require authentication, for example by providing the superuser password.

Remote Code Execution

Generally speaking, superuser access to a PostgreSQL database is equivalent to remote code execution. As a superuser, code execution can be achieved by abusing two features of Postgres. The first feature, “Large Objects,” effectively allows uploading arbitrary files while the second feature allows loading arbitrary (attacker-controlled) shared libraries into the remote Postgres process. Under Linux, a simple way for an attacker to execute arbitrary code when a malicious library is loaded is to mark a function containing that code as a constructor. Thereby, merely loading that library leads to arbitrary code execution without any further requirements.

In the context of GMS, attackers can obtain native code execution as the postgres system user by applying this technique. The procedure has been implemented as the first part of an exploit.

Escalating Privileges

After obtaining initial access as postgres to the appliance, opportunities for escalating privileges to root were explored. It quickly turned out that the directory /etc on the appliance is world-writable:

[postgres@gms ~]$ id
uid=104(postgres) gid=104(postgres) groups=104(postgres)
[postgres@gms ~]$ ls -lad /etc
drwxrwxrwt   17 root     root          1040 Feb  5 10:09 /etc
[postgres@gms ~]$ ls -la /etc/this_is_a_test
ls: /etc/this_is_a_test: No such file or directory
[postgres@gms ~]$ touch /etc/this_is_a_test
[postgres@gms ~]$ ls -la /etc/this_is_a_test
-rw-------    1 postgres postgres         0 Feb  5 10:09 /etc/this_is_a_test

While existing files cannot be modified or removed (due to the directory’s sticky bit), new files within /etc can be created by unprivileged users. As many privileged processes source their configuration files from /etc, manipulating these files is a frequently used technique for attackers to escalate privileges. One file that may be repurposed in that way and usually does not exist is /etc/ld.so.preload. Its purpose is to store a list of paths to shared objects that the dynamic linker automatically loads into the process whenever a dynamically-linked binary is run. This includes processes running as root, even if they are instantiated from root-owned SUID binaries, such as /bin/ping. The ownership of the shared object that is to be injected is not considered by the dynamic linker and therefore may be owned by any user. As a consequence, unprivileged users can inject arbitrary code into a root-owned process such as ping, thereby escalating privileges.

The privilege escalation has been implemented as the second step of the exploit.

Exploit

Both exploitation steps have been automated in the Python script sgms-rce.py and the shared object pwn.so. The Python script takes the hostname or IP address of the targeted GMS instance as a command line argument:

$ ./sgms-rce.py --help
usage: sgms-rce.py [-h] [--port PORT] [--dbname DBNAME] [--pwn-so-file-local PWN_SO_FILE_LOCAL]
                   [--pwn-so-path-remote PWN_SO_PATH_REMOTE] [--lpe-so-path-remote LPE_SO_PATH_REMOTE]
                   host

positional arguments:
  host

optional arguments:
  -h, --help            show this help message and exit
  --port PORT, -p PORT
  --dbname DBNAME, -d DBNAME
  --pwn-so-file-local PWN_SO_FILE_LOCAL
  --pwn-so-path-remote PWN_SO_PATH_REMOTE
  --lpe-so-path-remote LPE_SO_PATH_REMOTE

It then connects to the PostgreSQL database, uploads the precompiled shared object pwn.so and loads it into the postgres process. The shared object code determines the socket file descriptor that belongs to the database connection of the exploit and redirects all further output to it. It then renames the shared object to lpe.so, prepares /etc/ld/so.preload accordingly and finally executes /bin/ping which will run as root with lpe.so injected. The LPE code cleans up the created files and starts an interactive Bash shell, which has stdin/stdout/stderr redirected to the connection’s former PostgreSQL socket (via dup2()).

Indicators of Exploitation

During exploitation, multiple PostgreSQL features are abused in order to eventually execute attacker-controlled code within the postgres process. Therefore, one may expect artifacts of the attack to be found in the database (/var/lib/infobright_pg/pg_data/base) itself or the corresponding write ahead log (/var/lib/infobright_pg/pg_data/pg_xlog). However, this assumption heavily depends on the technical details of the exploitation steps. For example, the attached exploit neither leaves a trace in the database nor in the write ahead log. In contrast, the Metasploit module linux/postgres/postgres_payload adds a new large object to the database each time the exploit is executed and all exploit-related transactions are recorded in the write ahead log. Neither exploit code attempts to clean up any artifacts in the database (besides deleting the shared object from disk). The reason for the difference in observable artifacts is that the MSF exploit code forks into a new process before it executes e.g. the meterpreter payload. This way, the exploited postgres process is able to gracefully finish handling the client and thereby implicitly commits the database changes and eventually writes them to disk. The exploit attached to this post does not fork a new process. Instead, it directly invokes the execve syscall for /bin/ping. At that point, the postgres process image responsible for handling the client is replaced with that of /bin/ping. Because the transaction has not been committed up to the point that the process image is replaced, it will never be committed in the future either. The exploit we demonstrated therefore does not leave any artifacts in the PostgreSQL database.

As any exploit must at least temporarily store a shared object to the filesystem, malicious shared objects may be found on disk. Again, successful detection heavily depends on whether the file got deleted afterwards and whether the path used during exploitation was backed by volatile memory.

For rapid analysis of PostgreSQL data folders, we found it helpful to grep for strings that are more or less likely to exist in (malicious) shared objects but are usually not found in any legitimate data that the PostgreSQL database processes in the context of the SonicWall GMS application.

[user@work rootfs]$ grep -rP '(\x7fELF|\.got|\.plt|\.shstrtab|\.dynamic|\.text|\.init|mmap|mprotect|memcpy|dup2|libc\.so\.6)' var/lib/infobright_pg/
Binary file var/lib/infobright_pg/pg_data/pg_xlog/000000010000000000000001 matches
Binary file var/lib/infobright_pg/pg_data/base/12016/11774 matches
Binary file var/lib/infobright_pg/pg_data/base/1/11978 matches
Binary file var/lib/infobright_pg/pg_data/base/1/11774 matches
Binary file var/lib/infobright_pg/pg_data/base/12021/11978 matches
Binary file var/lib/infobright_pg/pg_data/base/12021/11774 matches

If the grep command returns a non-empty result, it is likely that someone attempted to exploit the database. Manual analysis of the identified files should be conducted to confirm or reject the result. If there are no hits, either no exploit attempts were made or the threat actor was stealthy and exploited the database before changes were persisted to disk.

Recommendations

In order to prevent the attack, SonicWall has released updates to their Global Management System product. According to their Security Advisory, a hotfix named GMS 9.3 Hotfix MAR-22474.1 is available to address the issue and should be applied to all vulnerable devices. If the fix cannot be applied, access to TCP port 5029 of affected devices should be blocked.

Additional Resources

Related Content