Capture Client 3.6 - Release Status and Availability
SuroopMC
SonicWall Employee
Earlier today, we released Capture Client 3.6 and enabled it for all of our Capture Client customers - this also means that attempts to login to Capture Client via Capture Security Center (CSC) or from inside MySonicWall (MSW) would automatically redirect you to the new portal.
However, soon after release we started seeing some performance issues:
- We're investigating some infrastructure issues that are preventing users from logging into the portal (either directly or via CSC or MSW)
- We're working with SentinelOne to investigate performance issues on their services that is impacting the experience of users who were able to login. This may also be delivering unexpected results in the older consoles (CC 3.1 and CC 3.5)
Also, for those endpoints that got auto-upgraded to Capture Client 3.6.24, you may be seeing the client as unlicensed. Please do not fret as this will not impact protection of your endpoints - they will still remain as protected as they were before the maintenance window.
We appreciate your patience and please rest assured that we are working hard to resolve the issue. We will provide an update as soon as the services are back online and usable.
Comments
@SuroopMC
https://captureclient-36.sonicwall.com/ for the new portal run into bad Gateway from csc
https://captureclient36.sonicwall.com/ ask to buy a new license.
The right portal is https://captureclient-36.sonicwall.com. The bad gateway is a symptom of the Infra issue we are seeing.
The portal should be back up now - again its https://captuerclient-36.sonicwall.com. Or if you prefer, just login via CSC.
Hi @SuroopMC nothing indicates more that a system is ready to go by constant red alert boxes showing "Request failed with status code 504". Is this really ready for production? It's slow like ever, spinning most of the time. It takes minutes for just showing the 5 Endpoints I have it on.
All systems are active, but the Console shows it as last active days ago, which is just wrong. I was in that situation before, hopefully I can get them uninstalled properly.
I just have it installed on a few internal systems so I'am somewhat relaxed on it if it's not breaking to much. But this is nothing I can recommend to a customer at the moment.
--Michael@BWC
Hi @SuroopMC
it was OK last night after you posted your "we're fine" message above. now it's the same as 24h ago. nothing works. 504 errors nothing else. NSM is also down or when able to login reports all firewalls down.
at least you finally changed the small note at the top of mysonicwall that you need another two days to fix this mess. As theres no other info at all expect this one note in here from last night.
What did we all learn re major issues handling -> not only technical fixing, but communication. Seems your dept. in charge of comm is on vacation...
dont expect to see this post survive, so dear Admin when you are deleting this post, at least think about it.
off to other things.
@Jürg good catch, I did not realized that the Banner got updated, so I stop wasting my time getting it to work for the next couple of days.
There is an Advisory up for this situation:
--Michael@BWC
I never actually saw that notification. It also doesn't help that Sonicwall sent a company wide email this morning, which sounds like the release is ready to go, which it clearly is not. "We are pleased to announce the SonicWall Capture Client 3.6 release".
I'm not even bothering opening a support case. I've had nothing but sub-par support for any of their product since I've started using it. It's always a race to close the ticket. If SW team is reading this, start focusing on the quality of the support and not tracking KPIs on how fast tickets are closed. That style of support fell out of fashion in the 90's.
My comment will most likely be removed. If thats the case, please take note that there are still issues going on and report back up the chain before removing:
Please resolve the issues before announcing general availability.
@jramsey same here.
one customer what has 5 licenses can wait few days. But the customer with dozens of licenses must be informed immedialty whats happening on the client-pcs. Therefore a functionable portal is an important tool to keep the overview over the Networkactivities.
I hope the Backoffice will get things done quickly...
--Thomas
Good Morning,
here is it exactly the same with 120 Endpoints:
Only two endpoints seems to be OK
Sorry, having my clients not under controll via console for me it is an big impact on security!!!
It is now the third day having this problems.
Greeting
Thomas
@SuroopMC what's the Status Quo on this? The Maintenance Banner is not shown anymore, does this mean everything should be fine?
As mentioned in the other threat, the one Endpoint I installed with 3.6 ended up twice and weird in the Devices Listing. Was this a glitch caused by some bugs or do I have to face this for the other endpoints as well. I would like to know what'll be the best next steps.
--Michael@BWC
@BWC - we are going to do a mass cleanup on our side to help resolve the issue. Please see the product advisory here https://www.sonicwall.com/support/product-notification/product-advisory-capture-client-3-6-upgrade-may-19-update/210518163702317/
If you are available to validate this in 2 hours, please send me a private message on this community with your Tenant Details (Tenant Name or Serial).
Of course - if you'd like to test with the manual workaround first per the advisory, please feel free.
Btw, @BWC - if your endpoints are still pointing to captureclient.sonicwall.com it means they haven't downloaded the 3.6.24 update yet. That is likely because you dont have a SonicWall-Managed policy. Please update your policy configuration in the OLD portal (captureclient.soniwall.com) to choose the 3.6.24 as part of your Capture Client policy - we recommend you set it to SonicWall-Managed for a better result.
Once that happens, your client should update - and you MAY get an "unlicensed" message on your client. Or you may get a duplicate entry in the 3.6 console. or both. Our script above will fix these issues if you don't have any endpoints in Static Groups.
Hi @SuroopMC thanks for the quick reply. I changed only one Policy at the moment to be SNWL managed because of this uncertain situation. Will repeat the steps for the other Policies when it got sorted out.
About the fix you mentioned, my Endpoints are in Static Groups so your fix will not apply?
The workaround for static assigned Endpoints to Groups states that the "old" Endpoint (shown as 3.1.5) should be Decommissioned and Deleted, which would leave the "new" Endpoint (shown as 3.6.24) available. How about this weird current user "srv_49XXXX-5D98-1F02-D562-68XXXXXXXX", anything of concern?
Why is the Visible Console IP changing all the time every few seconds,. It should be the Public IP of the Endpoint, not some random AWS EIP. While watching the Device Details it automatically changes every 5-10 seconds, showing my public IP and some EIP.
--Michael@BWC
@BWC - yes, because your endpoints are in static groups our fix wont apply. Technically, it can be run but then you would have to reconfigure your Static Groups. Hence the workaround - effectively our script is mass-executing the workaround to make it easier.
The srv_ user is specific for Servers because of a new fix for Server devices - nothing of concern.
Not sure about the Console Visible IP - is it supposed to be from your ISP?
@SuroopMC had all tenent prepared to 315 self managed.
hope he old portal is running over the weekend?
@SuroopMC it shows two Console Visible IP addresses, which is odd. The first one is my static public IP and the second IP is some AWS EIP which is changing every few seconds. Will send you some details in private.
--Michael@BWC
@ThK - yes it will be. Please review the advisory here - if you have Static Groups configured, the script will not work for you and you will have to apply the workaround suggested. https://www.sonicwall.com/support/product-notification/product-advisory-capture-client-3-6-upgrade-may-19-update/210518163702317/
Our customer has 80 devices, but almost all of them are offline on https://captureclient-36.sonicwall.com/ and show Version 3.1.5 even though the client is already on 3.6 and shows unlicensed. If I try to update them on https://captureclient.sonicwall.com/, it doesn't do anything. This launch was really bad, is Sonicwall still working on this issue?
@SuroopMC I updated two macOS Clients to 3.6.24 and at least one of them seems to work as intended. But this KB article is not 100% accurate.
It mentions to give SentinelOne Extensions the Full Disk Access, but there is no SentinelOne Extensions, in my Case the 4th Entry is SentinelAgent. I guess I have to use that instead?
The 2nd Client shows "Offline and protected" which is weird, because it shows as online in the Management Backend. How do I get this Endpoint Online?
--Michael@BWC
@SuroopMC Here is a summary of everything I am still seeing. We were a 35 customer. Per recommendations I made sure prior to update all policies were set with "SonicWall-managed Latest release" and that all clients were up-to-date with latest software. Per the Console this morning, this is still true:
Many agents still offline since the upgrade. A mixture of both Windows and Mac. Although one random S1 agent is one line (last line in purple)?
This might not me related to the upgrade, but random linux agents online, but last active over a month ago?
Then there is the situation where the client upgrade to 3.6.24, but the S1 agent has not upgrade. I see this on both Mac and Windows
I went to one of my testing machines for the situation above. Tons of log messages just repeating with:
Call to 'getPolicy' failed!
Failed to dump data to file: '_error', error '30'
Failed to update policy error '14' message ''
I tried to force some action on it in the console, but no success.
With that said, there are some clients that did upgrade successfully to 3.6 and have the correct S1 version with an online status. In addition, we pushed out a new 3.6 executable to one machine and it worked fine and came up with no issues.
And last nit picky thing, in the console, client policies page. "ENFORCED FIREWELLSS". What are FIREWELLSS? I assume that should be FIREWALLS?
Here is my experience so far hope it is helpful for anyone.
I am coming from CC 3.1 portal. Any updates I have done so far have not been an issue. Device may need to be rebooted before updating. The licensing and duplicate entries on the 3.6 portal are also not an issue.
Now the issues:
Looks like all my policies when migrated from 3.1 portal to 3.6 have enabled device control. This is an issue due to Hyper V virtual switch's being disabled by device control due to a known issue. There is a workaround which I have deployed on the systems that I have encountered this issue.
https://www.sonicwall.com/support/knowledge-base/how-to-enable-hyper-v-virtual-switch-adapter/200907080248627/
But what I also had to do was go into each tenant of mine and disable device control so I don't have any further issues when migrating them.
The other thing I have noticed is that there are still folders/files of the older version of S1 remaining on all my upgraded machines. Is this normal?
That being said previous to this release I was having issue with VSS backups. Support gave me S1 commands to run but didn't work on the older version. So now since the upgrade has taken place I have run those commands and they didn't resolve the issue. I have contacted support and updated the ticket but still haven't heard back (probably have their hands full right now with all the release issue's). Hopefully @SuroopMC gets back to me sooner rather than later.
About 2/3 of the clients upgraded on their own. I have been manually installing the 3.6 client over the 3.1 client on the remaining clients. When the upgrade finishes, it starts to install the S1 piece. On most of the workstations, I receive this message:
curl failed: error 28 (Timeout was reached): 'Operation too slow. Less than 1 bytes/sec transferred the last 30 seconds' I get the message where it retries 5 times then says it has stopped trying. Is this a S1 issue or will this fix itself?
@Jim356 - yes it should retry again later and the S1 client will be installed.
@Bret - the VEEAM issue should have been resolved with the latest agent we have (4.7.x). If it hasnt then I would recommend raising a case to have it investigated further.
Regarding the additional folders, they should get removed on the next reboot. Nothing to worry about.
@SuroopMC My issue is actually with Quest Rapid Recovery. That being said the latest available agent is 4.6.14.304 for windows.
I've had this case open with you since March
@jramsey - thanks for your detailed feedback.
@Bret - you're right, my bad on the typo. For windows and Linux it is indeed 4.6.x. For macOS it is 4.7.x.
@SuroopMC I have tried using 4.6.x and the issue continues. I then tried running the commands previously supplied:
- sentinelctl config -p agent.vssConfig.enableResearchDataCollectorVssWriter -v false -k "Agent passphrase"
- sentinelctl config -p agent.vssConfig.enableStaticResearchDataCollectorVssWriter -v false -k "Agent passphrase"
- sentinelctl config -p agent.vssConfig.agentVssWriters -v false -k "Agent passphrase"
The commands ran without a problem but the issue still continues. Is there anything else that needs to be done?
I have updated the ticket that has been open since March.
@SnoopW Thats for the feedback on the issues. Things are looking better.
All of the windows agents look good and seem to have upgraded themselves to 3.6 and the correct S1.
All the MacOS agents are still having issues. They show online in the 3.5 portal, but offline in 3.6. They still haven't upgraded, regardless of the correct policy being setup. It doesn't appear to be the oversubscription error you referenced above. Any known issues on this front?
@jramsey you rebooted your macOS Clients already? Over here the update went through after rebooting, running macOS 10.15. My own client was stuck yesterday in the state "Offline and protected" but it's Online after giving it a night of good sleep.
--Michael@BWC