WCF Services with URL Rewriter

Mar 18, 2010 at 7:40 AM
Edited Mar 18, 2010 at 7:55 AM

Hi, anyone have tested the URL Rewriter with WCF (.svc) proxy with IIS 5/6 ? I notice that nberardi in here said he can access svc while in my case I always get 404 error.

Error is similar to:

[EndpointNotFoundException]: The service '/Proxy/ServiceName.svc' does not exist.
at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
[HttpException]: The service '/Proxy/ServiceName.svc' does not exist.
at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.End(IAsyncResult result)
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.ExecuteSynchronous(HttpApplication context, Boolean flowContext)
at System.ServiceModel.Activation.HttpModule.ProcessRequest(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

As far as I am understand, this error is pretty similar to the one discussed in here and here. I have followed everything in readme and the first mentioned discussion but still can not get the proxy working for svc while I notice other things, like aspx seems to be working.

I even try rule like :

RewriteRule ^/Proxy/(.*).svcp$         http://otherserver/Application/$1.svc                       [NC,P,QSA]

 in which I try to use other extension (here is ".svcp") to get the page. With this I can get the html representation of the service (GET Method) or the wsdl, however, in actual application, it can not be used since WCF appears to check whether queried address as described in exception below:

[EndpointNotFoundException]:The message with To 'http://localhost/Proxy/ServiceName.svcp' cannot be processed at the receiver, due to an AddressFilter mismatch at the EndpointDispatcher.  Check that the sender and receiver's EndpointAddresses agree.

at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

My own investigation into UrlRewriter code (and altering it) found that somehow if request is made to a svc extension file, the ASP.NET pipeline executed is only Begin Request and AuthenticateRequest (which I added for testing). Surprisingly PostAuthenticateRequest event is not executed, which in my experience indicated that CompleteRequest is executed somewhere. However, I do not install other HttpModule and even disabling ScriptModule. In addition, since there is no other rewriting rule aside than proxied ones, there is almost no alteration done by RewriterModule's context_BeginRequest method. I have also tried to add

context.SkipAuthorization = true; 

to the context_BeginRequest, but I have no luck using this. Request to svc seems to bypass rewriter module completely aside than Begin Request and AuthenticateRequest , even if I delete the script mapping for .svc in IIS Application or default website.

Any workaround/solution for this matter?

Thanks beforehand.

 

 

Coordinator
Mar 18, 2010 at 3:23 PM

That is because the WCF handler is intercepting the request before the rewriter is.  Try moving things around on the httpHandler in the web.config or remove the *.svc all together.

Mar 22, 2010 at 11:39 AM
Edited Mar 23, 2010 at 9:15 AM

I am not able to disable wcf albeit I already modify the web.config with:

 

<httpHandlers>
   <remove verb="*" path="*.asmx"/>
   <remove verb="*" path="*.svc"/>
</httpHandlers> 

Previously it was

 

 

<httpHandlers>

<remove verb="*" path="*.asmx"/> 
<remove verb="*" path="*.svc"/>
<add verb="*" path="*.asmx" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/> <add verb="*" path="*_AppService.axd" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
<add verb="GET,HEAD" path="ScriptResource.axd" type="System.Web.Handlers.ScriptResourceHandler, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" validate="false"/>

</httpHandlers>

 

I also try to remove wcf through ServiceModelReg.exe -k but I am also not able to make the proxy work.

Somewhere seems something are blocking the proxy. Still something making the module stops after AuthenticateRequest. I even try to move other events contents to beginrequest so it would be processed there without any luck. Seems that is due to "context.Handler = proxy;" in context_PostMapRequestHandler() . Is there any consideration to NOT to call into the proxy handler directly/manually with ProcessRequest in proxy handler?

EDIT: Sorry, formatting which seems to be lost. Just notice it later.

Coordinator
Mar 22, 2010 at 2:54 PM

I cannot read that, can you please fix it up.

Mar 23, 2010 at 9:31 AM
Edited Mar 23, 2010 at 9:32 AM

 

As mentioned in my previous comment, I add the following in context_BeginRequest()

if
(context.Request.Path.ToString().EndsWith(".svc"))
{

context_AuthenticateRequest(sender, e);
context_PostAuthenticateRequest(sender, e);
context_AuthorizeRequest(sender, e);
context_PostAuthorizeRequest(sender, e);
context_ResolveRequestCache(sender, e);
context_PostResolveRequestCache(sender, e);

// check to see if this is a proxy request if (context.Items.Contains(Manager.ProxyHandlerStorageName))
{
IHttpProxyHandler proxy = context.Items[Manager.ProxyHandlerStorageName] as IHttpProxyHandler;

string pathAndQuery = proxy.ResponseUrl.PathAndQuery;

string proxyPath = context.Request.ApplicationPath;
if (proxyPath != "/" && proxyPath != string.Empty)
{
// if not installed in root we must remove the proxy path first. int index = pathAndQuery.ToLower().IndexOf(proxyPath.ToLower());
if (index > -1)
{
pathAndQuery = pathAndQuery.Remove(index, proxyPath.Length);
}
}

context.RewritePath("~" + pathAndQuery);
context.Handler = proxy;
proxy.ProcessRequest(context);

}

context_PostRequestHandlerExecute(sender, e);
context.ApplicationInstance.CompleteRequest();

}

Basically it forces the rewriter module to take over the process and end it before any other thing take it over. After this code is added the .svc can be proxied, and not returning 404 error.

I don't know the side effect aside bypassing ASP.net security.

Nevertheless seems to get the proxy to work with WCF, it seems that I need to modify "POST" request (which contains the url requested) due to exception: 

[EndpointNotFoundException]:The message with To 'http://localhost/Proxy/ServiceName.svcp' cannot be processed at the receiver, due to an AddressFilter mismatch at the EndpointDispatcher.  Check that the sender and receiver's EndpointAddresses agree.

at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

From wireshark, I found that the POST request of WCF seems to contain a parameter a:To which contains the original url. Is URL Rewriter capable to modify POST request?
Coordinator
Mar 23, 2010 at 12:53 PM

The Proxy is capable of modifying the POST.  Take a look there.  Also I don't think this is completely necessary.  Have you just tried removing the WCF HTTP Module in the web.config.  To see all the HTTP Modules running, you can set a break point in the begin request and use look through all the following array:

HttpContext.Current.ApplicationInstance.Modules

There might be one related to the service you are missing.  You should remove any that you know you don't need, then the rewriter should proceed smoothly without any custom code.

Mar 24, 2010 at 6:59 AM
Edited Mar 24, 2010 at 12:43 PM

This is my List of Modules in HttpContext.Current.ApplicationInstance.Modules

  1. OutputCache
  2. Session
  3. WindowsAuthentication
  4. FormsAuthentication
  5. PassportAuthentication
  6. RoleManager
  7. UrlAuthorization
  8. FileAuthorization
  9. AnonymousIdentification
  10. Profile
  11. ErrorHandlerModule
  12. ServiceModel
  13. RewriterModule
  14. DefaultAuthentication

I do not see any Module related to WCF there, and there is no WCF Http Module in web config. As far as I understand, it seems WCF service lives outside ASP.Net albeit it can share things to ASP.Net.

This link seems to confirm my understanding, as it said "HttpModule extensibility: The WCF hosting infrastructure intercepts WCF requests when the PostAuthenticateRequest event is raised and does not return processing to the ASP.NET HTTP pipeline. Modules that are coded to intercept requests at later stages of the pipeline do not intercept WCF requests."

However, on the same link further down, it mentions that there is compatibility mode to allow WCF lives as ASP.Net module with aspNetCompatibilityEnabled in web.config. I managed to test it and it enables the proxy request by adding

 

<system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" />
  </system.serviceModel>

 

into web.config. I believe this is alternative way aside than adding custom code, but I don't know whether this one is better or not, as I still stuck with POST problem, wrong regex probably as I used:

RewriteRule ^/Proxy/(.*).svc$         http://otherserver/Application/$1.svc                       [NC,P,QSA]

And that one is for full string with prefix "/Proxy/" and suffix ".svc", Need something which can modify middle one in POST contents, and hoping that WCF security does not have something which check this part with some hash code/token verification or anything similar.

 

EDIT:

Now I am able to use WCF with the URLRewriterProxy. Basically aside above steps (The additional code or the aspNetCompatibility) I need to reimplement new type of rule (which is in my case I name it "InRewriteRule", which is mostly similar to "OutRewriteRule".

This rule is used to modify POST contents, so the WCF POST request to parameter is replaced with the proxied one. The original URL Rewriter is not able to modify this request, as it can only modify the header, not the contents (which is exist in InputStream property)

Main thing I add:

(in SendRequestToTarget() of ProxyHandler.cs)

           // ContentLength is set to -1 if their is no data to send
            if (request.ContentLength >= 0 && !knownVerb.ContentBodyNotAllowed)
            {
                StringBuilder requestFromClientContent = new StringBuilder();
                int bufferSize = Manager.Configuration.Rewriter.Proxy.RequestSize;
                using (Stream bufferStream = new BufferedStream(context.Request.InputStream, Manager.Configuration.Rewriter.Proxy.BufferSize))
                {
                    byte[] buffer = new byte[bufferSize];

                    try
                    {
                        while (true)
                        {
                            // make sure that the stream can be read from
                            if (!bufferStream.CanRead)
                                break;

                            int bytesReturned = bufferStream.Read(buffer, 0, bufferSize);

                            // if not bytes were returned the end of the stream has been reached
                            // and the loop should exit
                            if (bytesReturned == 0)
                                break;

                            string requestFromClientContentPart = Encoding.UTF8.GetString(buffer, 0, bytesReturned);
                            requestFromClientContent.Append(requestFromClientContentPart);
                        }


                        
                        byte[] bufferToTarget = Encoding.UTF8.GetBytes(requestFromClientContent.ToString());

                        bufferToTarget = Manager.RunInputRulesDirectly(context, bufferToTarget);
                        request.ContentLength = bufferToTarget.Length;

                        //Actual Sending of request
                        Stream requestStream = request.GetRequestStream();
                        requestStream.Write(bufferToTarget, 0, bufferToTarget.Length);
                    }
                    catch (Exception exc)
                    {
                        Manager.Log("Error on request: " + exc.Message, "Proxy");
                    }
                }
            }
 
Basically this part of code will buffer request, rewrite the request (using "Manager.RunInputRulesDirectly(context, bufferToTarget); " ) and then do the actual request.
Coordinator
Mar 24, 2010 at 1:36 PM

There is a module named "ServiceModel" and you are modifying a web.config option called "system.serviceModel" I think they may be related.

Mar 25, 2010 at 10:22 AM

Based on your suggestion, I tried to add <remove name="ServiceModel"/> into <httpModules> and it achieves  the same effect with the aspnetcompatibility and the code change (which means proxy call is successful).

It means that ServiceModel is indeed the name of WCF (well the dll namespace name is also similar, but I don't notice it previously), nevertheless it is added by something into the IIS mechanism (by force), as if you removing that <remove name="ServiceModel"/> then suddenly WCF intercept the HTTP pipeline again.

Wow, now it means that we have 3 alternatives doing this. As side note, I also don't see other thing referencing serviceModel in my web.config. That ServiceModel segment is newly added.

 

However, the POST modification is still needed, and due to URL Rewriter can not modify POST contents, and the InRewriteRule still needs to be implemented. Alternatively you can change the service code by adding [ServiceBehavior(AddressFilterMode=AddressFilterMode.Any)] in the service.

I am now using aspNetCompatibilityEnabled and POST input rewriting. I also believe the InRewriteRule may be also useful in some other situation for other people similar to OutRewriteRule. By the way, for improving compatibility with encoding, the previous code is now changed to:

            // ContentLength is set to -1 if their is no data to send
            if (request.ContentLength >= 0 && !knownVerb.ContentBodyNotAllowed)
            {
                StringBuilder requestFromClientContent = new StringBuilder();
                int bufferSize = Manager.Configuration.Rewriter.Proxy.RequestSize;
                using (Stream bufferStream = new BufferedStream(context.Request.InputStream, Manager.Configuration.Rewriter.Proxy.BufferSize))
                {
                    byte[] buffer = new byte[bufferSize];

                    try
                    {
                        bool rewriteInput = false;
                        Stream requestStream = null;

                        //Only if Input Rule is exist, this require some modification to get the number of rules from RuleContext
                        if (Manager.InputRuleCount(context) > 0)
                        {
                            //Only Rewrite Input for known contentType
                            if (context.Request.ContentType != null)
                            {
                                if ((context.Request.ContentType.ToLower().Contains("application/soap+xml"))
                                    && context.Request.ContentType.ToLower().Contains("charset=utf-8"))
                                {
                                    //Only rewrite soap request
                                    rewriteInput = true;
                                }
                            }
                        }

                        if(!rewriteInput)
                        {
                            request.ContentLength = context.Request.ContentLength;
                            requestStream = request.GetRequestStream();
                        }


                        while (true)
                        {
                            // make sure that the stream can be read from
                            if (!bufferStream.CanRead)
                                break;

                            int bytesReturned = bufferStream.Read(buffer, 0, bufferSize);

                            // if not bytes were returned the end of the stream has been reached
                            // and the loop should exit
                            if (bytesReturned == 0)
                                break;

                            if(rewriteInput)
                            {
                                string requestFromClientContentPart = Encoding.UTF8.GetString(buffer, 0, bytesReturned);
                                requestFromClientContent.Append(requestFromClientContentPart);    
                            }
                            else if (requestStream != null)
                            {
                                requestStream.Write(buffer,0, bytesReturned);
                            }
                            
                        }


                        if (rewriteInput)
                        {
                            //Buffer and rewrite
                            byte[] bufferToTarget = Encoding.UTF8.GetBytes(requestFromClientContent.ToString());
                            bufferToTarget = Manager.RunInputRulesDirectly(context, bufferToTarget);
                            request.ContentLength = bufferToTarget.Length;

                            //Actual Sending of request
                            requestStream = request.GetRequestStream();
                            requestStream.Write(bufferToTarget, 0, bufferToTarget.Length);
                        }
                        else
                        {
                            
                        }
                    }
                    catch (Exception exc)
                    {
                        Manager.Log("Error on request: " + exc.Message, "Proxy");
                    }
                }
            }

Coordinator
Mar 25, 2010 at 7:27 PM

I will have to look in to the an input processing rewrite rule.  However that is only useful in situations where the content body is present in the HTTP request.  Like a POST.  So it has very limited use.  And to be honest you should have to be doing everything that you are doing.  Because the point of a proxy is to mimic an external browser like Firefox or IE.  And obviously the requests work with out any modification from Firefox or IE.  So you just have to figure out what the difference is so that I can make sure it is supported in the proxy.  Also the aspNetCompatability attribute is because WCF isn't just for web based communication, it can be used for binary communication between windows apps too.  So that is why that extra flag is there.  

 

Basically if you setup the proxy website to not have the ServiceModel module in it everything should flow through just fine.  And then you can determine what isn't getting sent through based on what request comes in from the browser, and what the request looks like as the proxy makes it.

Mar 26, 2010 at 3:13 AM
Edited Mar 26, 2010 at 3:27 AM

Yup , no problem, I know that I must do what I have been done anyway since it is pretty much specific (WCF), and I have added something about Out Rewrite Rule anyway (add and forcing buffering, to handle my own problem with chunked encoding, where the rewriting is not done due to the should be rewritten string is in different chunk) .

Getting input processing rewrite rule for contents in POST message is nice anyway (^^).

Thanks anyway for your help, your software is great, especiallyconsidering you have done pretty much everything yourself.