Rewrite to a local Apache instance?

Nov 6, 2008 at 10:58 AM
Edited Nov 6, 2008 at 11:00 AM
Hi,

I have a very basic question for a reverse proxy: I'm running both IIS and an Apache on a Windows 2003 (well, Windows Home Server). I would like to expose parts of the Apache instance (which runs on a proprietary port) via IIS.

So i would like to forward requests to https://my.homeserver.com/subversionAliasOnIIS to https://my.homeserver.com:3443/MySubversionRepoOnApache/

Is that scenario supported?

Thanks in advance,
Christian
Coordinator
Nov 6, 2008 at 12:39 PM
RewriteRule ^/subversionAliasOnIIS  https://my.homeserver.com:3443/MySubversionRepoOnApache/ [NC,P]
Nov 6, 2008 at 4:09 PM
I would prefer not to hard code the domain information... 

RewriteCond %{HTTP_HOST} (.+)
RewriteRule ^/subversionAliasOnIIS http://%0:3443/MySubversionRepoOnApache/ [NC,P]

One thing to take note is the flags at the end of the rule.  If you want to proxy the request then you would want to include the [P] option.  If you want to redirect the connection then you would want to replace the [P] with a [R] option.

Have a look at the mod_rewrite documentation for a more complete description of options.

Coordinator
Nov 6, 2008 at 5:42 PM
@dscoduc

Remember you are using a version that isn't available to to the general public yet which includes conditional variables.  Plus it would be the following when version 2.2 is released.  The conditional variables start at "1";

RewriteCond %{HTTP_HOST} (.+)
RewriteRule ^/subversionAliasOnIIS http://%1:3443/MySubversionRepoOnApache/ [NC,P]
Nov 6, 2008 at 6:16 PM
You're right, I forgot.  But that leads me to an important question...  Are you planning on posting the build versions onto the CodePlex Source Code page?  I haven't seen any versions of the code posted and would like very much to keep up with your development...  And who knows, you might find others (like me) who want to write additions or improvements to help your project along...
Coordinator
Nov 6, 2008 at 11:10 PM
I wish there was a way to do a push from my TFS system to the codeplex system.  I might create a custom build script to do this at a later time, that way I can only push when all tests have passed.  Either way it should be pretty soon.  Contact me if you want the source before than Chris, I have a pretty extensive modeling system to allow you to create custom rules that would be hard or impossible to do in the Apache syntax.
Feb 23, 2009 at 7:29 PM
I have the exact same scenario,

All requests to http://www.myexternalsite.net/repository should be redirected to http://localservername:20199/repository 

This is my rewrite rule file:
-------------------------------------
RewriteEngine On
RewriteRule ^/repository/(.*) http://localservername:20199/repository/$1 [P]
-------------------------------------

(I haven't removed the server name hard coding or introduced the case insensitive option yet, sorry. First things first.)

Using a web browser to the URL works perfectly, including HTTP Basic Authentication. I am impressed!

Well.. my other client, a TortoiseSVN Repository Browser, that uses WebDAV, doesn't work!

It seems WebDAV uses "HTTP OPTIONS" requests, and those are not proxied to the local server with the above rewriting rule, I'm afraid.

This is a request I sniffed:

-------------------------------------
OPTIONS /repository HTTP/1.1
Host: localservername
User-Agent: SVN/1.5.5 (r34862) neon/0.28.3
Keep-Alive:
Connection: TE, Keep-Alive
TE: trailers
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
-------------------------------------

And this is the response:
-------------------------------------
HTTP/1.1 405 Method Not Allowed
Date: Mon, 23 Feb 2009 19:43:36 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 3456 
-------------------------------------

I was expecting any errors to be returned from Apache at this point.

Please help me. Best regards, Fred
Coordinator
Feb 23, 2009 at 10:35 PM
Do you know if this error is coming from my Url Rewriter or from IIS 6.0.  IIS 6.0 has disabled many of the WebDav options because of the security risk associated with them.  If ASP.NET and IIS will accept the HTTP Method my proxy will relay it. 

Also try enabling logging,

RewriteLog c:\{my website path}\log
RewriteLevel 9

And see if you get anything from this about the URL.  If the URL is going through my URL rewriter it will be at least registered in the log.

Nick
Feb 25, 2009 at 6:24 AM
Thanks for the quick and good reply!

Yes, perhaps it the configuration of IIS6.0 that is giving me trouble.

I enabled WebDAV on IIS, and some other permission settings that supposedly is required. (I looked at http://blogs.iis.net/robert_mcmurray/archive/2008/03/04/enabling-webdav-on-iis-6.aspx and did all the things in the script, but using Control Panel and IIS Manager. I confess WebDAV is not my area of expertise.)

I also added the logging lines you suggested. I write the log to c:\temp and gave Everyone Modify rights to that folder temporarily.

The results?

- The same symptoms.

The .NET URL Rewriter log is created but remains empty no matter if I user a browser (that still works) or the other client (that still gives me the error).

The IIS access log shows:
--------------------------------
2009-02-25 06:45:38 OPTIONS /repository - 80 - SVN/1.5.5+(r34862)+neon/0.28.3 405 0 0 (Method not allowed)
2009-02-25 06:45:38 OPTIONS /repository - 80 - SVN/1.5.5+(r34862)+neon/0.28.3 405 0 0 (Method not allowed)
2009-02-25 06:45:38 OPTIONS / - 80 - SVN/1.5.5+(r34862)+neon/0.28.3 200 0 0 (OK)
2009-02-25 06:45:38 PROPFIND / - 80 - SVN/1.5.5+(r34862)+neon/0.28.3 207 0 0 (Multi-Status WebDAV)
2009-02-25 06:45:38 PROPFIND / - 80 - SVN/1.5.5+(r34862)+neon/0.28.3 207 0 0 (Multi-Status WebDAV)
2009-02-25 06:46:38 GET /repository - 80 - Internet+Explorer+8 301 0 0 (Moved Permanently)
2009-02-25 06:46:38 GET /repository/ - 80 - Internet+Explorer+8 401 5 0 (Unauthorized)
2009-02-25 06:46:47 GET /repository/ - 80 - Internet+Explorer+8 200 0 0 (OK)
--------------------------------

The Apache log shows:
--------------------------------
 - [25/Feb/2009:06:46:38 +0100] "GET /repository/ HTTP/1.1" 401 401 (Unauthorized)
username [25/Feb/2009:06:46:46 +0100] "GET /repository/ HTTP/1.1" 200 646 (OK)
--------------------------------

Any clues on how to make it work?

Best regards,
Fredrik
Coordinator
Feb 25, 2009 at 1:44 PM
One last log I need to see before, I can tell where this error is coming from, so that I can help you out:

As suggested in my last post, please enable the URL Rewriting Log:

RewriteLog c:\{my website path}\log
RewriteLevel 9

This will help me determine if my URL Rewriter is handling the request or if it is getting rejected before hand.  Also here is the code for the Proxy:

http://www.codeplex.com/urlrewriter/SourceControl/changeset/view/27780#364389

A few lines under the SendRequestToServer method you will see where I am just doing a straight copy of the HTTP Method.  So if something is getting denied it is either happening in the System.Net.HttpWebRequest or before the request even reaches my URL Rewriter.
Feb 25, 2009 at 3:05 PM
Nick, thank you again. Well structured code also, I like it!

As you told me from the beginning, I also now think that this request doesn't even reach your code. It seems there is some configuration on IIS needed. I wish I knew which configurations...

I mentioned in the last reply that I added your log setting and gave Everyone Modify permissons on the folder I selected. The log file is created but I'm afraid is is empty, not even the successful requests from the Web browser is logged.

Best regards, Fredrik
Coordinator
Feb 25, 2009 at 4:27 PM
Edited Feb 25, 2009 at 6:03 PM
This is totally my fault, I had you using a folder for the logging instead of a file path:

RewriteLog c:\{my website path}\log\log.txt
RewriteLevel 9

In my haste to put up some logging information for you, I forgot the file part of the path.
Coordinator
Feb 25, 2009 at 4:28 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.
Feb 25, 2009 at 7:55 PM

You've already been a lot more helpful than I could have expected. Thank you!

Although in my efforts to get it to start logging, I already tried to name the log file "C:\temp\rewriter.log" to no avail. Note that I do restart IIS after every single configuration change to make sure the rules file is re-read. So in short, still no entries in the rewriter log file.

I have an idea for a walkaround to the original problem: perhaps I can use an SSL connection from the client to the Apache server. In that case, any contents of the http packet such as if it is WebDAV or not should not matter to the .NET URL Rewriter, frankly there is no way for it to find it out even. Does this seem reasonable?

If so, would you mind teaching me what demands there would be on the connection? I assume both the IIS that listens to port 80 and Apache, that listens to port 20199 in my case, both must have separate valid SSL certificates for www.myexternalsite.net. True? Anything else I should know.

Best regards, Fredrik

Coordinator
Feb 25, 2009 at 8:58 PM
Well the key to getting the rewriter log to work is that the path needs to be running under the same permissions as IIS.  Sometimes giving access to everyone or all users doesn't work.  In most cases giving access to "NETWORK SERVICES" with full permissions to the folder will allow IIS to write to the file system.

To be honest I don't think it is a connection between the client and the Apache server.  I think it has something to do with IIS and or .NET.  Do you have access to the full headers when the 405 Method Not Allowed is sent?  Becuase there should be a header called "Allow" which says which methods are allowed. 

Another possibility that I have seen before is that a tool called UrlScan

http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1697

Blocks these headers by default.  You would probably know if you had this installed, but if you want to check.
  1. Go to the properties of your website
  2. Look under ISAPI Filters
  3. Let me know if there is anything on this screen.
Feb 26, 2009 at 5:56 AM
Edited Feb 26, 2009 at 6:58 AM

Why the log file is created (which makes be belive it has permissions) and not filled with data, even though the proxying works when surfing, is a mystery. I gave IIS_WPG, which includes NETWORK SERVICE, Full Permissions but it didn't do any difference. I also checked, and the Application pool really does run under the NETWORK SERVICE identity.

My previous suggestion on the connection between the Client and Apache is regarding a way to "bypass" the security mechanisms of IIS, my theory being that if the connection is encrypted, IIS can't stop any requests on the grounds of the verb or of any WebDAV peculiarities, because IIS wouldn't have access to that encrypted information. I don't really know, though, how an SSL connection is proxied (or are there two separate connections in such a case?).

I guess I do have access to the full headers since I can sniff the network. Would you mind explaining again which particular information you are interested in? Is it in the OPTIONS request or the reply for example?

URLScan was a good try, but it is not installed.

The Web Site has no ISAPI Filters. "Aspnet_isapi.dll" is a wildcard application map on the "repository/" virtual directory that is in question.

Best Regards,
Fredrik
(PS. Perhaps I should mention that connecting the TortoiseSVN Repository Brower directly to the Apache works of course.)

Coordinator
Feb 26, 2009 at 11:16 AM
Edited Feb 26, 2009 at 11:19 AM
Fredrik,

I think I have it.

According to HTTP specification when a 405 Method Not Allowed is sent it should also send a header called "Allow" which contains the allowed verbs:

RECV MESSAGE HEADER HTTP/1.1 405 Method Not Allowed
Allow: OPTIONS, TRACE, GET, HEAD
Content-Length: 1564
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Tue, 30 Sep 2008 20:59:44 GMT


And I bet that the VERB's coming back in the Allow header match those in the ".axd" extention.  Which you can find on this page where you define the VERB's allowed for each file extension.



In the RewriterModule

http://www.codeplex.com/urlrewriter/SourceControl/changeset/view/27780#364436

I am setting the path to RewriteProxy.axd before running the proxy.  This is occurring on context_PostResolveRequestCache.

So maybe if you go in and enabled all VERBS for .axd that will fix your issue.
  1. Go to properties of your website
  2. Go to tab Home Directory
  3. Click Configuration button
  4. Select ".axd" under Application Extensions
  5. Click Edit button
  6. Select All Verbs
  7. Click OK
I think this may be the problem, at least my fingers are crossed that it is the problem.  Let me know.
Feb 26, 2009 at 6:33 PM
I'm really sorry to say that I still get the error message. It seemed so perfect, but no it didn't work.

I don't really know were to look now, and I understand it is even harder for you. Thank you for all your kind help!

Best regards
Fredrik
Coordinator
Feb 26, 2009 at 8:11 PM
You wouldn't happen to have a copy of IIS 7 on Windows 2008 or Windows Vista available?

Because I tried the following code snippet, and everything worked.

        HttpWebRequest request = WebRequest.Create("http://blog.stackoverflow.com/wp-admin/css/login.css") as HttpWebRequest;
        request.Method = "OPTIONS";
        HttpWebResponse response = request.GetResponse() as HttpWebResponse;

So I doubt that .NET is blocking it, and my URL rewriter definitely isn't setup to block requests.  It must be at the server level.  If you feel up to continuing, I am game to stick it out with out until the end.  :)
Mar 4, 2009 at 7:42 PM
Edited Mar 4, 2009 at 7:44 PM
Hello again,

Well the whole point is my installing it on my Windows Home Server, which we can regard as a Windows Server 2003 in these matters.

I finally corrected an error and got the logging to work. (I had mistakenly written "RewriteLevel" instead of the correct "RewriteLogLevel".)

It was nice to see logging and thanks to it I discovered something:

It seems the client strips any trailing slashes from the URL. So, www.myexternalsite.net/repository/ becamomes www.myexternalsite.net/repository. My rule was like this: RewriteRule: ^/repository/(.*) http://localservername:20199/repository/$1 [P] which didn't match, because of the missing trailing slash. Changing the rule to RewriteRule: ^/repository(.*) http://localservername:20199/repository$1 [P] (note the removal of ending slashes in the rule) made it match. I assume there is a prettier way to do this.

So what happens now. Here's the UrlRewrter log, trying to access sub-directory "pk".

-------------------------
Rule Processing: RewriteLogLevel: 9
Rule Processing: RewriteRule: ^/repository(.*) http://localservername:20199/repository$1 [P]
**********************************************************************************
Rewrite: Input: http://www.myexternalsite.net/repository/pk
Rule 0: Input: /repository/pk
Rule 0: Rule Pattern Matched
Rule 0: Output: http://localservername:20199/repository/pk
-------------------------

A match! Promising!!!!

There is still something left to do though. After having done a successful HTTP BASIC authentication, the TortoiseSVN client "hangs". (This does not happen if I try connecting to the localservername directly.)

So a leap towards success. Any tips on where to look next?

Best regards,
Fredrik
Mar 12, 2009 at 10:21 PM
Edited Mar 12, 2009 at 10:27 PM

Hello again,

I've been sniffing the network and this is how it should work it seems:

1. client does a "PROPFIND /repository HTTP/1.1"
2. the internal apache server responds with a "HTTP/1.1 207 Multi-Status"
3. a series of requests/responses of the above kind continues

When going through the URL Rewriter, the reponse in 2. is never returned to the client using the setup mentioned in this thread. Allow me to give details on this response that isn't proxied back.

-----------------------------------
Transmission Control Protocol, Src Port: 20199 (20199), Dst Port: abbaccuray (1546), Seq: 2202, Ack: 1942, Len: 579
HTTP/1.1 207 Multi-Status
Server: Apache/2.2.11 (Win32) DAV/2 SVN/1.5.5
Content-Length: 404
Content-Type: text/xml; charset="utf-8"

<?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:" xmlns:ns0="DAV:">
   <D:response xmlns:lp1="DAV:" xmlns:lp2="http://subversion.tigris.org/xmlns/dav/">
      <D:href>/repository/!svn/vcc/default</D:href>
      <D:propstat>
         <D:prop>
            <lp1:checked-in><D:href>/repository/!svn/bln/126</D:href></lp1:checked-in>
         </D:prop>
         <D:status>HTTP/1.1 200 OK</D:status>
      </D:propstat>
   </D:response>
</D:multistatus>
-----------------------------------

I really hope this helps you help me. Any ideas why it isn't proxied back?

Best regards,
Fred

Coordinator
Mar 13, 2009 at 1:51 AM
Quick question.  Is the proxy handling the request back and forth.  Or are all the requests making it to the browser? 
Mar 13, 2009 at 9:28 AM

Client <--> IIS (Url Rewriter configured as a proxy as described before) port 80 <--> Apache port 20199

The response that I told about was sniffed when I looked at the communication between the Client and Apache directly without IIS involved.

It isn't returned to the client, however, in the scenario I am trying to solve.

Thank you for your time and expertice, Nick.

Regards,
Fredrik

Coordinator
Mar 13, 2009 at 11:28 AM
To start analyzing this.  I need 3 sets of results. 

1. The correct requests without the proxy.
2. The Client - Proxy requests.
3. The Proxy - Server requests.

So that I can see what is actually making it through.  This will help me figure out what is making it through the proxy and what isn't which really helps in tracking down the problem.  If you want to take this off line so you can e-mail me the results just go to http://www.managedfusion.com/info/contact-us.aspx and send me a message.

Nick