c# - Pinpointing cause of slow WCF self-hosted service in certain environments -

c# - Pinpointing cause of slow WCF self-hosted service in certain environments -

i'm trying debug why our software (client , server) slow in environments. i've tried few things without success , i'm @ loss how go on tracking downwards issue. question is: things can seek or resolve issue?

we developing wcf self-hosted service run on server, , client application connect it. service interacts database (ms sql server).

setup

server running in vm (2 gb ram, 1 core vm) client separate desktop machine on same intranet client startup (authentication , init -- 40 service calls) done in less 5 seconds

setup b

server not virtualized (16 gb ram, dual xeon e5-2640) client running on same machine server (connect 127.0.0.1) test purposes client startup takes 45 seconds

similarities

server , client software same version both setups server operating scheme windows 2008 r2 windows role of file server , iis (there shouldn't other server-type activities going on except sql server) windows feature of .net 4.5.1

differences

setup b using sql server 2012. setup using sql server 2008 setup , b on 1000 miles apart (different networks, managed under different ia rules) some differences in windows server features, nil looks should cause problem

things i've tried

setup has no firewall , no antivirus on server. setup b has windows firewall disabled. symantic endpoint protection running; insight, bloodhound, auto-protect, download insight disabled testing made no difference. sql integration, reporting , analysis services disabled little change. i've looked in windows event viewer , don't see out of ordinary, i'm not sure if there's specific should looking for. i've enabled wcf logging (as per here), looking @ log in svctraceviewer.exe isn't telling me useful. likewise, attempting log various stats in perfmon has not been productive.

other thoughts

not using entity framework in service. file storage on same physical machine server running iis related webtool, wcf self-hosted service should unaffected (right?) running sql profiler, of database calls resolving (milliseconds) don't think it's database issue. cpu utilize low. ram usage on setup b less 60% (per task manager). since doesn't seem database or resource issue, i'm guessing it's networking or wcf issue.

things i'm going seek next

there few differences still left resolve -- different patches, server features, etc clone setup vm onto setup b server not sure

service app.config

<?xml version="1.0" encoding="utf-8"?> <configuration> <configsections> <section name="foonet.serviceprovider" type="foo.foonet.serviceregistration.classes.serviceproviderconfiguration, foo.foonet.serviceregistration" /> <section name="nlog" type="bar.com.foo.nlog.config.configsectionhandler, foo.nlog" /> </configsections> <connectionstrings> <add name="basedb" connectionstring="..." /> <add name="userstore2" connectionstring="..." /> <add name="userstore" connectionstring="..." providername="system.data.entityclient" /> <add name="metriccollectiondal.properties.settings.massmetricsconnectionstring" connectionstring="..." /> </connectionstrings> <appsettings> <add key="maxrequestlength" value="16384" /> <add key="exclusiveflag" value="true" /> <add key="subscriptiondeletenotification" value="true" /> </appsettings> <nlog xmlns="http://www.nlog-project.org/schemas/nlog.xsd" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"> <targets> <target name="file" xsi:type="file" filename="${basedir}/foonetlog.txt" keepfileopen="true" layout="${date:format=mm/dd/yyyy hh\:mm\:ss}|${level}|${windows-identity}|${callsite}|${message}" archiveevery="day" maxarchivefiles="5" /> <target name="session" xsi:type="file" filename="${basedir}/sessionlog.txt" keepfileopen="true" layout="${date:format=mm/dd/yyyy hh\:mm\:ss}|${level}|${windows-identity}|${callsite}|${message}" concurrentwrites="false" deleteoldfileonstartup="true" /> </targets> <rules> <logger name="*" minlevel="error" writeto="file,session" /> </rules> </nlog> <foonet.serviceprovider xmlns="foo.foonet"> <plugins directory="plugins" /> <baseaddress uri="https://localhost:6040/serviceprovider" /> <brokers count="1" remoteregister="true" registrationinterval="1" /> </foonet.serviceprovider> <system.servicemodel> <diagnostics performancecounters="all" /> <services> <service behaviorconfiguration="serviceproviderbehavior" name="foo.foonet.serviceregistration.classes.serviceprovider"> <endpoint address="" binding="ws2007federationhttpbinding" bindingconfiguration="ws2007binding" behaviorconfiguration="endpointbehavior" contract="foo.foonet.shared.server.interfaces.iserviceprovider" /> <endpoint address="mex" binding="mexhttpsbinding" contract="imetadataexchange" /> <host> <baseaddresses> <add baseaddress="https://localhost:6040/serviceprovider" /> </baseaddresses> <timeouts closetimeout="00:10:00" opentimeout="00:10:00" /> </host> </service> </services> <behaviors> <servicebehaviors> <behavior name="serviceproviderbehavior"> <servicemetadata httpgetenabled="true" httpsgetenabled="true" policyversion="policy15" /> <servicedebug includeexceptiondetailinfaults="false" /> <servicesecurityaudit auditloglocation="application" suppressauditfailure="false" serviceauthorizationauditlevel="failure" messageauthenticationauditlevel="failure" /> <servicethrottling maxconcurrentcalls="32" maxconcurrentsessions="20" maxconcurrentinstances="2147483647" /> </behavior> </servicebehaviors> <endpointbehaviors> <behavior name="endpointbehavior"> <dispatchersynchronization maxpendingreceives="6" /> </behavior> </endpointbehaviors> </behaviors> <bindings> <ws2007federationhttpbinding> <binding name="2007mtombinding" closetimeout="01:00:00" opentimeout="00:40:00" receivetimeout="01:00:00" sendtimeout="00:40:00" transactionflow="false" maxbufferpoolsize="0" maxreceivedmessagesize="2147483647" messageencoding="mtom" usedefaultwebproxy="false"> <readerquotas maxdepth="2147483647" maxstringcontentlength="2147483647" maxarraylength="2147483647" maxbytesperread="2147483647" maxnametablecharcount="2147483647" /> <reliablesession ordered="false" inactivitytimeout="01:00:00" enabled="false" /> <security mode="transportwithmessagecredential"> <message establishsecuritycontext="false"> <claimtyperequirements> <add claimtype="http://bar.com/foo/foonet/claims/firstname" isoptional="false" /> </claimtyperequirements> </message> </security> </binding> <binding name="ws2007binding" closetimeout="01:00:00" opentimeout="00:05:00" receivetimeout="01:00:00" sendtimeout="00:40:00" maxreceivedmessagesize="2147483647" maxbufferpoolsize="0" usedefaultwebproxy="false"> <readerquotas maxdepth="2147483647" maxstringcontentlength="2147483647" maxarraylength="2147483647" maxbytesperread="2147483647" maxnametablecharcount="2147483647" /> <reliablesession inactivitytimeout="01:00:00" /> <security mode="transportwithmessagecredential"> <message negotiateservicecredential="false" establishsecuritycontext="false"> <claimtyperequirements> <add claimtype="http://bar.com/foo/foonet/claims/firstname" isoptional="false" /> </claimtyperequirements> </message> </security> </binding> </ws2007federationhttpbinding> <wshttpbinding> <binding name="authenticationbinding" closetimeout="01:00:00" receivetimeout="01:00:00" sendtimeout="00:40:00" maxreceivedmessagesize="2147483647" bypassproxyonlocal="true"> <readerquotas maxdepth="2147483647" maxstringcontentlength="2147483647" maxarraylength="2147483647" maxbytesperread="2147483647" maxnametablecharcount="2147483647" /> <reliablesession inactivitytimeout="01:00:00" /> <security mode="transport" /> </binding> </wshttpbinding> <ws2007httpbinding> <binding name="securityauth" sendtimeout="00:40:00" maxreceivedmessagesize="2147483647" closetimeout="01:00:00" receivetimeout="01:00:00" usedefaultwebproxy="false"> <readerquotas maxdepth="2147483647" maxstringcontentlength="2147483647" maxarraylength="2147483647" maxbytesperread="2147483647" maxnametablecharcount="2147483647" /> <reliablesession inactivitytimeout="01:00:00" /> <security mode="transportwithmessagecredential"> <message negotiateservicecredential="true" /> </security> </binding> </ws2007httpbinding> </bindings> </system.servicemodel> <startup> <supportedruntime version="v4.0" sku=".netframework,version=v4.0" /> </startup> </configuration>

update 2014-11-18

received "snapshot" of server (single file.vhd). created new vm in hyper-v (had add together hyper-v role) on server b single cpu , 2 gb ram , loaded in server vhd. local vm of server localhost connection fast expected. thing tells me there's nil wrong physical box. tried changing number of cores allocated vm 4 little no change.

stood server vm. made 3 changes

ip address visible on network joined local domain changed service business relationship [previous] domain local computer (not domain) account

i ran wireshark , captured initial authentication , startup. instantly got ton of entries following:

i changed business relationship runs service business relationship on domain, couldn’t communicate @ all. exception (on client) says

“the communication object, system.servicemodel.channels.servicechannel, cannot used communication because in faulted state.”

which seems indicate there’s uncaught/unhandled exception.

managed pull out more helpful exception:

“security back upwards provider interface (sspi) authentication failed. server may not running in business relationship identity 'host/[computer]'. if server running in service business relationship (network service example), specify account's serviceprincipalname identity in endpointaddress server. if server running in user account, specify account's userprincipalname identity in endpointaddress server.”

was start trying figure out means (e.g. here), changed domain business relationship “local scheme account” checkbox in services.msc , super speedy.

c# wcf