Overview

You may sometimes experience connections timing out while connecting to a node in a ScaleArc cluster from a SQL client, or even an administration tool such as Microsoft SQL Server Management Studio (SSMS).

This might also be accompanied with entries such as this,

CID: <X>, Client IP: 172.xx.xx.xx, User: Application, Debug_Code: 527, Message: Connection to server failed due to error :'Cannot assign requested address' [Bound local socket to 172.xx.xx.xx:52261] portChosen:52261. Retrying SSL: No , State: 42, SSID: 18793938, DB: testdb1, DB IP: 172.yy.yy.yy, Type: 37
YYYY-MM-DD HH:MM:SS

in the Alert Log of the cluster.

Solution

Diagnosis

One of the reasons for this could be that ScaleArc is not able to open any new connections to the database. This can be proven by running the following command and checking if the number of already open connections is close to the maximum limit ScaleArc is allowed to keep.

sudo netstat -alpn | grep idblb_mssql_X | wc -l

X is the numeric cluster ID in the above command, visible on the Clusters page below the name of the cluster. Run this for all the clusters and check if the total number is close to the limit for any cluster as any one cluster could affect the performance of the other clusters. Note that the command above is applicable for a cluster of MS SQL Servers.

Additionally, it can also be observed that connecting with a SQL client directly to the database works, but the attempt times out while trying to connect to ScaleArc.

Steps To Fix

Restart the cluster that is nearing the limit of maximum allowed open connections from the UI by clicking on the Stop button & starting it up again.

Testing

Once the cluster has restarted, check if the connections are going through to ScaleArc from the SQL client.