Porting normal, salted MD5 hashes to htpasswd format for auth (Serv-U=>Apache)

Greetings!

I am new here, and have a (hopefully) interesting question about Apaches MD5 hashing. I have to say, that I am not a developer, so i can understand Apache source code only in a very, very, very limited way.

What I want to do:

Port Users of a Serv-U 6 FTP Server including their hashed passwords over to an Apache password file (like the ones created by the htpasswd tool).

Operating system: Win32 for both the FTP- and Webserver.
Apache Version: 2.2.4
Password Encryption: Supposedly MD5 for both the FTP- and Webserver.

Main Problem: Apaches representation (encoding) of the MD5 hash as far as I understand this.

First, let's assume we have a User called "testuser" with his password also being "testuser". I have created all Hashes using these plaintext credentials for testing purposes.

Ok, when hashing that password, the Serv-U 6 FTP server gives you this in its ServUDaemon.ini in the [USER=testuser|2] section:

Password=qi481F86E841C9D742BA6CC4261CF8EACF

The first two characters are the salt, the rest is the MD5 Hash represented in what is clearly Hex. I verified this by trying to bruteforce that hash using the tool "mdcrypt" with a very limited challenge alphabet. Like this:

mdcrack-12.exe -s testuser -v -b qi -M MD5 481F86E841C9D742BA6CC4261CF8EACF
"-s" specifies the alphabet, "-b" specifies the salt as prefix.

This will use only the characters contained in the string "testuser" plus the salt "qi" for the challenge. On my machine, the crack attempt was successful after only 11 seconds using these parameters. So this is clearly a valid MD5 hash using a 2-character prefixed salt. The hash is generated over the string "qitestuser".

Ok, so much for the Serv-U FTP server.

Now to the interesting part, the Apache 2.2.4 webserver. I generated a hash using htpasswd, forcing it to MD5 encryption (should be default in Win32 anyway, but just to make sure). What htpasswd creates, is this:

testuser:$apr1$oo1.....$7Qr1pbvQE4uKH8gpveF.h/

What i found out is, that the syntax for this string is obviously:

<username>:$apr1$<salt>$<md5 password hash>

However, this is no MD5 password hash like I have ever seen before. So I found out (actually a collegue informed me about it), that there are several representations - or encodings - of an MD5 hash being used out there. One is hexadecimal, like the one of Serv-U. However, encoding might also be base64 (see http://www.garykessler.net/library/base64.html), ASCII, or something else.

So, this cannot be regular base64 I thought (after reading, what base64 actually is), because some Hashes like this one contain a "." character, which is not included in the base64 alphabet. It also cannot be ASCII, there should be more special characters if so.

So, it is not standard base64, not base32 obviously, not ASCII and not HEX encoding. It is some smaller alphabet though, since there are only letters, capital letters, numbers and "/" and "." in those Apache MD5 hashes.

I tried to read Apache-Win32 htpasswd source code, and found a few things in the files included in htpasswd.c and in htpasswd.c itself.. Does this mean, that it uses its own alphabet!? There are so many md5-related files, I don't know where to start.. See:

In htpasswd.c:


This seems to be some sort of customized base64 encoding? Maybe i just don't understand that code correctly...

So, bottom line and primary question: How can i convert a salted standard MD5 hash represented in HEX to something i can give Apache to authenticate against using basic auth? (Or: How can i convert said MD5 hash to the format generated by the htpasswd tool)

Thanks a lot!

Edit:

Also found this piece of information:

"MD5 $apr1$ + an Apache-specific algorithm using an iterated (1,000 times) MD5 digest of various combinations of a random 32-bit salt and the password"

This doesn't sound so good to me..

Welcome to Dev Shed.

Shouldn't you be using htdigest to generate your user/pass files for Apache when you're using MD5 digest authentication?

Thanks for the reply!

Actually, i'm currently using "AuthType basic" authentication with the AuthUserFile directive pointing to the file, where the credentials are stored, not digest auth. Something like this for instance:



...with "C:/Apache2/authfiles/htpasswd.txt" containing something like the above "testuser:$apr1$oo1.....$7Qr1pbvQE4uKH8gpveF.h/" string.

I'm using SSL for the data connection, so i thought, basic auth should be sufficient, cause nobody can sniff the credentials anyway? The above entry comes directly from my httpd.conf, slightly modified. This is used to secure a Subversion Repository, but securing normal directories works almost in the same way, but of course you know that already...

Does Apache2.2 use "normal" MD5 hashes when using Digest Auth?! I guess not, since the htpasswd tool has no options to create a "normal" MD5 hash...

Thanks!

Why do you assume that the password stored in a password file created by htpasswd would be MD5ed? Did I miss that somewhere in the documentation?

I would assume, though, that the password in a password file created with htdigest would be MD5ed, as the documentation seems to imply this.

That's why:

"C:\Server\Apache 2 Webserver\Apache2\bin>htpasswd
Usage:
htpasswd [-cmdpsD] passwordfile username
htpasswd -b[cmdpsD] passwordfile username password

htpasswd -n[mdps] username
htpasswd -nb[mdps] username password
-c Create a new file.
-n Don't update file; display results on stdout.
-m Force MD5 encryption of the password (default).
-d Force CRYPT encryption of the password.
-p Do not encrypt the password (plaintext).
-s Force SHA encryption of the password.
-b Use the password from the command line rather than prompting for it.
-D Delete the specified user.
On Windows, NetWare and TPF systems the '-m' flag is used by default.
On all other systems, the '-p' flag will probably not work."

Documentation suggests, that this MD5 algorithm is not really very "standard". See Apache Documentation. I'm not allowed to post URLs, but it's on a page called "password_encryptions.html". Sorry for trying to post an URL before, should have read the forum rules more carefully.



Of course, I could use plaintext passwords for new Apache users that have to auth.

But the idea is to create a Website for the Users of my FTP server, where they can see published FTP statistics. This area should be password protected. Only FTP users should be allowed to see that website, and they should auth with their FTP credentials. Those credentials are MD5ed (at least, the password is). That's why I want to port those credentials of the FTP server over to Apache 2.2.

So Apache isn't using a basic MD5 of the password but is MD5ing a salted result in a loop through their own algorithm. I think it'd be pretty difficult to migrate that over without replicating that process in your migration scripts.

Note, however, that the password created by htdigest is a little easier to create:6f59b1de73da6859309c9357845e46bc is the MD5 result when passing bar:foo:baz, where baz is the password.

I wonder if you could use the salt from your FTP's password file as the realm to create a matching MD5 sum?

Tried that, but the resulting htdigest Hash is different from what Serv-U produces. And I found a Post on the web, where somebody successfully reconstructed the htdigest Hashing in Python:



If I'm not mistaken, this means, that htdigest uses the username, the realm and the password for the hash creation. A pity really, cause that was quite close i guess, the htdigest Hash seems to be simple MD5 with HEX encoding..

I have now posted this issue in a second forum, and a developer suggested that he might implement support for the standard MD5 hash that Serv-U uses (directly in Apache) if it isn't too much work for him.

Another guy suggested to place a PHP file into the directory, and make the PHP file handle the auth directly against the Serv-U configuration file. Might also work... Hmm, will take a few days, maybe even longer, who knows. But if a solution comes up in the other forum, I'll post it here, and vice-versa!

A solution looking forward (both for you and others who may visit who haven't implemented anything yet) is to use database backed authentication, assuming Serv-U offers a module that will talk to a database.

Serv-U can use ODBC for auth, and it can also auth against a Windows Domain (LDAP). The former might be the best solution, but it still depends on how the data is stored in the backend (hashes again?). I didn't check that yet. I don't want to auth against a Windows domain or local system users, since I don't want to have those FTP/Web users as system users at all (plus I don't have a domain set up).

Let's see, decision making will take a bit of time, still waiting for that developer who said he might modifiy the Apache APR module to support standard salted MD5.

I'm not sure how Serv-U handles its ODBC, but you specify the query in Apache. So you SELECT user FROM table WHERE pass = ?, so you can also WHERE pass = MD5(?), etc.