SQL Linked Server Query failed with “Login failed for user …”

SQL Linked Server Query failed with “Login failed for user …”

Rate This
  • Comments 75

SQL Server 2005: “Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'”.

SQL Server 2000: "Login failed for user "(null)". Reason: Not associated with a trusted SQL Server connection".

 

The errors listed above is very typical when deploy linked server with delegation. They actually are thrown by the linked server and pass by middle server to the client application. In this post, I will discuss how to properly configure SQL instances and Windows environment in most common scenario and try to make configuration steps as explicit as possible.

 

By using delegation in distributed query, such as linked server query, the SQL instance obtains impersonated token of the user logon credential to gain access to resources of another SQL instance, the linked server. In delegation setting, the client connection and linked server object are configured to use integrated authentication in SQL Server’s term as opposed to SQL login. Some time integrated authentication also referred as trusted connection or Windows authentication. Linked server login can also use SQL login, but it is not discussed here.

 

To simplify the discussion, let’s assume two SQL Server instances are installed on machine A and B respectively. Also, let’s assume A is the middle server that has a linked server object configured which points to a SQL instance on machine B. If the client is on machine C different from A, we call it double-hop setting; if the client is collocated with middle server machine A, we call it single-hop setting. In single-hop setting, it is relatively straightforward to configure linked server to work. Believe or not, double-hop setting requires more careful configurations as you will see. This is because in single-hop setting, windows NTLM authentication, which is available in most common setting if all machines are windows, is sufficient for delegation; while in double-hop setting, Kerberos authentication is mandate for flowing user’s credential through machine boundaries from the client to the linked server. It requires windows domain, correct DNS name resolution, proper account setting in both Active Directory and SQL Server.  To make sure Kerberos delegation [1] is correct becomes vital to operate distributed query with delegation. The authentication scheme required by delegation in different setting is illustrated by the following table.

 

Authentication scheme

C to A

A to B

Single hop

NTLM or Kerberos

(C is on the same box as A)

NTLM or Kerberos

Double hops

Kerberos

Kerberos

 

[Create Linked Server Object on Middle Server]

Before getting into details on how to configure other components, since I am talking about delegation in the context of SQL Server distributed query, let’s first give an example on how to configure a linked server object on A and set up its login to use delegation.  To do so, you need the following two steps.

 

(1) Use sp_addlinkedserver to create a linked server object and name it “LinkedServer” which points to the SQL instance on machine B, SQLB.

 

“EXEC sp_addlinkedserver @server=’LinkedServer’,

                                                 @srvproduct=''”,

                                                 @provider='SQLNCLI',

                                                 @datasrc=’SQLB’,--the data source

                                                 @provstr="Integrated Security=SSPI; "

To verify if the command is executed correctly, run query

select * from sys.servers where name='LinkedServer'”;

 

(2) Use sp_addlinkedsrvlogin to configure login to use self-mapping as following

exec sp_addlinkedsrvlogin ‘LinkedServer’, 'true'”

Step (2) makes middle server A try to use impersonated token of user to authenticate to server B. To verify that the linked server is setup for “self-mapping”, run query

 

select uses_self_credential as delegation

from sys.linked_logins as L, sys.servers as S

where S.server_id=L.server_id

            and S.name=N'LinkedServer'

 

The resulting table should show the delegation column is “1”.

 

[Test Linked Server Query in Single-hop Settting]

Before test-drive a link server query in single-hop setting, you need also make sure that the client user can make direct query to the SQL instances on both A and B. This means that the user account, either windows domain account or a machine account, must have permission to access both SQL instances.

 

(3) To verify the user domain account has permission to access both SQL instances, use your favorite client tool, for example,“osql –E –S SQLA” and “osql –E –S SQLB”. If you are failing for whatever reason, please refer to [5][6].

 

(4) To test linked server query, run query at SQLA,

“select * from LinkedServer.master.dbo.sysdatabases”.

 

 

[Configure and Test Double-hop Setting]

To deploy delegation based linked server in double-hop setting, the followings need to be configured correctly.

(3)   Kerberos in Windows mandates Windows domain. Therefore the user account needs to be a domain account and middle server and linked server need to join a domain. All machines involved in the delegation, including client machine, middle server and linked server, must have good TCP/IP connectivity between each other and to the domain controller and Active Directory. To not complicate thing further, we assume that A, B and C are in same Windows domain D and the user account is a domain account in D.

(4)   The user’s domain account must NOT select “Account is sensitive and cannot be delegated” in its Active Directory properties of domain D. Please refer to [1] on how to configure this on Active Directory machine.

(5)   The service account under which the SQL instance is running must be “trusted for delegation”, configured in Active Directory. If the service is running as “NT AUTHORITY\SYSTEM” or “NT AUTHORITY\NETWORK SERVICE”, the computer must be “trusted for delegation”. Please refer to [1] on how to configure this on Active Directory Machine. You need to have domain admin privilege to do/verify so.

(6)   The user domain account must have permission to access both SQL instances from C. To verify, use your favorite client tool, for example,“osql –E –S SQLA” and “osql –E –S SQLB”. If you are failing for whatever reason, please refer to [5][6].

(7)   If the SQL connections are to use TCP/IP connectivity, configure and verify that SQL connections from C to A and A to B are using Kerberos authentication. Please refer to [2] on how to configure Kerberos for SQL. In a nutshell, both services on machine A and B need to have a SPN created in Active Directory for SQL service. If the service running account is not “NT AUTHORITY\SYSTEM”, you need to configure the SPN on the Active Directory Machine with domain admin privileges. To verify that every hop is using Kerberos and TCP connectivity, run query

 

“select net_transport, auth_scheme from sys.dm_exec_connections where session_id=@@spid”

 

when (a) connect to A from C (b) connect to B from A. To make Kerberos work for both hops is crucial and some time it might not be very straightforward on what goes wrong. If fail to get Kerberos to work, please (a) verify If the SPNs are configured and well-formed according to [4]; use “setspn –L acccoutname” to verify the SPN (b) verify if DNS reverse lookups of both machine A and B return well-formed FQDNs. Use “ping –a machinename” on machine A, B and C to verify DNS works as expected, i.e. returning FQDN; (c) make sure that there is no cached Kerberos ticket on machine A and C. Use “klist purge” to purge all tickets. There might be delay before Windows local security authority (LSA) requests a new ticket from Active Directory. Sometime, you need to log out and log back in again before a new Kerberos ticket can take effect. For more Kerberos troubleshooting techniques, please refer to [3].

(8)   If the SQL connections are to use Named Pipe connectivity, SQL level Kerberos is not required as opposed to TCP connectivity. This is because Windows named pipe protocol can use Kerberos to authenticate logon user under the cover. You need to verify that both machine SPNs “HOST/machinename” and “HOST/ machineFQDN” is well-formed use “setspn –L machinename”.

 

(9)   Since double-hop can have combination of Named pipe connectivity on one hop and TCP on the other, the following table is valid configuration for delegation. Run query

 

“select net_transport, auth_scheme from sys.dm_exec_connections where session_id=@@spid”

 

when (a) connect to A from C (b) connect to B from A.

 

{net transport, auth_scheme}

C to A

A to B

1

NP, NTLM*

NP, NTLM*

2

TCP, Kerberos

NP, NTLM*

3

NP, NTLM*

TCP, Kerberos

4

TCP, Kerberos

TCP, Kerberos

* Note that even SQL shows NP is using NTLM at SQL level, Windows named pipe protocol can still use Kerberos under the cover ( the SMB/CIFS windows file system, upon which the named pipe is implemented, can use Kerberos) that allows Kerberos delegation.

 

After verifying the above 9 steps, finally we come to the point to run a double-hop linked query. If you still encounter issues after follow these steps, please send me the error message, your configuration, the output of the queries listed for every step and information required by [6].

 

In discussion above, we assume the A,B and C are in the same domain. If they are in different domains, apart from verifying 9 steps listed above, you also need to make the domains involved trust each other. 

 

Note that setspn.exe, klist.exe referred in this post can be downloaded as part of Windows resource kit. Kerberos delegation is only supported on windows 2000 and onward.

 

[Reference]

[1] Troubleshooting Kerberos Delegation. http://www.microsoft.com/technet/prodtechnol/windowsserver2003/technologies/security/tkerbdel.mspx

[2] Using Kerberos with SQL Server. http://blogs.msdn.com/sql_protocols/archive/2005/10/12/479871.aspx

[3] Troubleshooting Kerberos Errors. http://www.microsoft.com/technet/prodtechnol/windowsserver2003/technologies/security/tkerberr.mspx

[4] http://support.microsoft.com/kb/889615/en-us

[5] http://support.microsoft.com/?id=840219

[6] http://blogs.msdn.com/sql_protocols/archive/2006/04/21/581035.aspx

 

 

Nan Tu

Software Design Engineer, SQL Server Protocols

Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights

 

Leave a Comment
  • Please add 1 and 4 and type the answer here:
  • Post
  • Nan,

    Thank you for the article.  Can you talk a little bit about when you have trusted domains.  Our cluster resides in a "Domainlet" and we are using trusted domains.  We are having some problems getting our linked server to use domain authentication.

    Matt
  • In this post, I focus on how NTLM and Kerberos are applied when connecting to SQL Server 2005 and try

  • Hi,

    on the end of stage [Create Linked Server Object on Middle Server] - I got 0 and not 1, any idea of what should I do now?

  • Hi Nan,

    Thank you very much for your article, it's really  interesting and it has helped me to understand a lot of things about linked servers. Anyway, I'm still having problems.

    I'm trying to link different servers in different domains: Prod and Dev (some of them are SQL 2000 and some SQL 2005) using double-hop.

    Between servers placed in Dev domain I don't have any problem, but when I try to link servers from Dev to Prod or viceversa or between servers in Prod, I receive the next message for SQL Server 2000:

    OLE DB provider "SQLNCLI" for linked server "192.168.1.11" returned message "Communication link failure".

    Msg 10054, Level 16, State 1, Line 0

    TCP Provider: An existing connection was forcibly closed by the remote host.

    Msg 18452, Level 14, State 1, Line 0

    Login failed for user '(null)'. Reason: Not associated with a trusted SQL Server connection.

    and this one for SQL Server 2005:

    Msg 18456, Level 14, State 1, Line 1

    Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.

    When I execute the query: "select net_transport, auth_scheme from sys.dm_exec_connections where session_id=@@spid"

    the response is: TCP - KERBEROS

    I don't have a clue what's happening, could you help me, please?

    Thanks a lot,

    Radamante

  • Note that this error:

    OLE DB provider "SQLNCLI" for linked server "192.168.1.11" returned message "Communication link failure".

    Msg 10054, Level 16, State 1, Line 0

    TCP Provider: An existing connection was forcibly closed by the remote host.

    means that the remote host RESET the socket from client to server.  This can happen for example if domain2 does not trust domain1 and IPSEC is enabled in your domain.  IPSEC can automatically RESET untrusted socket attempts.

    The other two errors are consistent with double hop failures.

    With these types of errors the basic rule of thumb is be systematic and eliminate one leg at a time.  Since you can get "TCP - KERBEROS" I am assuming this is from the first leg, so now you need to troubleshoot the second leg.

    Start on the middle SQL Server and use OSQL or SQLCMD to attempt to connect to the next SQL, does it work?  Log onto the middle SQL Server and connect to SQL locally, then try linked server query, does this work?  Etc...

  • I am still experimenting, but it seems on clustered instances it's necessary to have each physical node's computer accounts in AD have the "trusted for delegation" as well as the service's user account.  

  • I have tried this query on sql 2005 (select net_transport, auth_scheme from sys.dm_exec_connections where session_id=@@spid)... it works.

    On SQL 2000, it fails not knowing the sys.dm_exec_connections

    Can you  give me a SQL2000 query to get the needed result?

  • What about Linked server login using SQL login?

  • OK I get it, I have to enter Remote login and password and not map anything

    I get the list of remote tables in Enterprise manager but select from "main" serevr to linked servers tables returns "The operation could not be performed because the OLD DB provider 'SQLOLEDB' was unable to begin a distributed transaction

  • Hiya,

    I am doing a double-hop and I followed all the steps up to point 7 where I was to run query:

    “select net_transport, auth_scheme from sys.dm_exec_connections where session_id=@@spid”

    when (a) connect to A from C (b) connect to B from A.

    All steps prior to this were set up correct.  But when I run this query I get this for both stages:

    net_transport   auth_scheme

    -------------   -----------

    Shared memory   NTLM

    Help!

    Ta

    Wendy

  • Try running dbcc freesystemcache('all') to free up pooled connections on the server side, then reconnect again.  I think what is happening is you are picking up a pooled connection that you made locally.

  • Hiya,

    Tried this and am now getting:

    From Client to Server1:

    net_transport   auth_scheme

    ----------------- ---------------

    TCP                 KERBEROS

    From server1 to Server2:

    net_transport   auth_scheme

    ----------------- ---------------

    Shared memory   NTLM

    Any ideas?

    Thanks

    Wendy

  • Wendy,

      I think your case is a single hop setup, i.e. the two server is on the same box. On the same box, shared memory and NTLM are given priority and your linked server query should be ok.

  • I have 10 servers.  

    8 are SQL 2000

    2 are sql2005 (A,B)

    2 are Clustered Sql 2000 (C,D).

    I have Query Analyzer on my desktop (Q).

    Single hop use of the linked servers works in all cases.

    Connecting through my desktop (Q):

    through server 'A' to double-hop to all servers works...except the clusters (C,D)

    'B' can use linked servers to all servers including the clustered pair (C,D)

    Why would the clustered servers provide the dreaded Msg 18452 to Server 'A' and not to server 'B'?

    I have carefully followed all the prescriptions in your fine article including checking the net transport.  I have carefully checked the Linked servers (and rebuilt them).

    Can you give me a pointer?

  • Hi, Steve

       The case is interesting. Can you provide more detail about the linked server deployment? Basically, how do you configure linked server from C,D to A,B? You client query running on Q, so it was double-hop, correct? and whether you double checked that Q->C, C->A or Q->D , D->A were using Kerberos over TCP? What is the configuration difference of A and B(SQL 2k5), whether they are same SKU? same default instance? Whether A and B are on the same box? With same OS?

    Thanks!

    Ming.

Page 1 of 5 (75 items) 12345