AT&T Wifi Calling problems
DDI
Newbie ✭
I have a TZ600 running 6.5.4.7-83n.
Clients connected to the LAN wirelessly via Engenius ECW220 nodes, tied in through Engenius Switches, to the TZ600 and out through a Residential Comcast Modem.
LAN->WAN traffic works fine. Have some Crestron controllers in the LAN that are accessible from the WAN that work fine.
The problem is that the iPhones don't have a reliable connection to AT&T Wifi calling. It's not a Wifi issue - they can hit other sites w/o issue. The Wifi signal is strong, and when testing the client was sitting about 5' from the wifi node, and not moving. There's little to no cell signal in the area, so it's not a strong cell signal that's intruding and causing issues. Private addressing disabled on the iPhones. Cell-Assist disabled on the iPhones.
When they call out, they usually get a connection through AT&T's wifi call server. Sometimes they don't, and 30 sec later it works.
When people try to call them, the phone doesn't ring - goes straight to voice mail.
Texts out from the client to somewhere outside the LAN work. Texts in to the client from somewhere outside the LAN work - mostly - MMS doesn't work.
Wifi Calling uses UDP Ports 4500, UDP 500, TCP 143. All these are set up as a service group, along with some other protocols that show up as an IPSEC NAT (ISAKMP, IKE, etc).
AT&T publishes various IP addresses on their website which are required, but monitoring has revealed that more than what they publish are needed - they've been added to the list of addresses for the rules.
We see two errors logged:
#1)
Ethernet Header
Ether Type: IP(0x800), Src=[3e:0d:a6:c8:f6:d6], Dst=[01:00:5e:28:00:01]
IP Packet Header
IP Type: UDP(0x11), Src=[192.168.0.77], Dst=[107.125.34.51]
UDP Packet Header
Src=[4500], Dst=[4500], Checksum=0x0, Message Length=128 bytes
Application Header
Not Known:
Value:[1]
DROPPED, Drop Code: 206(l2 mcast but dest ip is unicast(#1)), Module Id: 25(network), (Ref.Id: _1828_uyHtRcemgvKpkv) 1:1)
This shouldn't be picked up as a multicast address, 192.168.0.77 is an iPhone on the LAN connected thru one of the Engenius nodes. I suspect this packet is when the iPhone is trying to tell the AT&T wifi calling server that it's available to receive calls. All of the packets get dropped.
#2)
Packets FROM 107.125.34.51 to 10.0.0.33 get dropped due to "Policy"
10.0.0.33 is the X1 IP address assigned to it by the Comcast modem. Set it up as a "static IP" in the Comcast modem, and then DMZ'd the Comcast modem to that address, with the internal Comcast firewall disabled.
We've tried numerous versions of Access Rules, and NAT policies. The NAT policies don't get hit. Access rules out for other traffic besides that which is dropped above get hit. Return rules from the WAN back to the LAN never get hit. We've tried setting the allowed addresses up (published by AT&T) as FQDN's - some would resolve, sometimes, sometimes they wouldn't.
I tried multiple DNS resolver settings (208.67.222.222, 208.67.220.220, 1.1.1.1, 1.0.0.1) - made no difference. I finally set up multiple Address objects with the IP's that the FQDN's resolved to, and combined them into a Group - the rules use the groups. Still no effect on the issue.
Support tried setting up an IP Helper for a multicast on port 4500 to some random multicast address that's used by the mDNS bonjour helper, and it passed some traffic, but didn't help the issue.
Anyone got any ideas?
Thanks in advance...
Answers
Have you looked at the above setting? Anytime I've run into this the above has helped. You may also consider adding exceptions to any security services for the network the IPhones are on.
Hi @DDI ,
I hope you are safe and well!
As per the issue description, I understand that you are facing the issue, only with iPhones. Can you please try the below and test?
• Please follow the steps suggested under: https://www.sonicwall.com/support/knowledge-base/unable-to-call-via-apple-wifi-calling/170505913456806/ . If you are still facing the issue, then try the suggestion below.
• Can you please try creating an access rule from LAN to WAN with service-IKE and increased the UDP Inactivity timeout to 300 seconds? (Even if this resolves the issue, please get in touch with technical support and ask if keeping it at 300 seconds is fine or there is a lower recommended value).
Note: Please make sure that you have taken a backup of your firewall and exported the settings to your local machine, before making any changes. Also, I would suggest scheduling a downtime and then test the steps above, just to be on a safe end. In case, you already have an open ticket with the support team, please update them about the changes made before they start troubleshooting further.
Regards
Karan
Knowledge Management Senior Analyst at SonicWall.
IKE Passthru is configured. DPI is off for the rules for the iPhones. Security services were set up to "except" the IP addresses of the iPhones. Didn't make a difference.
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
I have no idea where the l2 mcast errors could be originating at - especially when the Sonicwall is claiming that 192.168.0.77 is a multicast address, and that the destination of 107.125.50.51 is somehow a problem. UDP packet, port 4500. I suspect that these are the packets which are reauthorizing the phones with IKE keys so they can use wifi calling.
Tried rebooting the iphones, have IGMP and MLD snooping configured on all the switches throughout the property.
Hello @DDI ,
I'm sorry to hear about this inconvenience. Please PM your case number so that we can follow up with our support team internally.
Regards
Karan
Knowledge Management Senior Analyst at SonicWall.
Thanks KaranM, I just did that...
-- David
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
I have no idea where the l2 mcast errors could be originating at - especially when the Sonicwall is claiming that 192.168.0.77 is a multicast address, and that the destination of 107.125.50.51 is somehow a problem. UDP packet, port 4500. I suspect that these are the packets which are reauthorizing the phones with IKE keys so they can use wifi calling.
Tried rebooting the iphones, have IGMP and MLD snooping configured on all the switches throughout the property.
I have a case open (43590888), have attached numerous comments and packet captures and no one's reviewed anything. Ticket status is "Waiting for Support". I've called in 2x, first time I waited on hold for 18 minutes, the Tech didn't solve the problem after maybe 2 hours. 2nd time I called in, I got through in about 5 minutes, spent another 2 hours, Tech didn't solve the problem. Tried calling a 3rd time this evening, waited on hold for 20+ minutes, then the system hung up on me (I have an Android phone).
How do I get the hardware swapped out? I've spent about 30 hours of my time on this, and there's no way I can bill my client for it. I need this resolved, or I'm going to have to yank out the Sonicwall, and then try to get a refund from my vendor. Go with some other device, or build my own with a Linux box. This is sad. Worst problem I've had with one of your firewalls in all the years I've been using them.
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
I have no idea where the l2 mcast errors could be originating at - especially when the Sonicwall is claiming that 192.168.0.77 is a multicast address, and that the destination of 107.125.50.51 is somehow a problem. UDP packet, port 4500. I suspect that these are the packets which are reauthorizing the phones with IKE keys so they can use wifi calling.
Tried rebooting the iphones, have IGMP and MLD snooping configured on all the switches throughout the property.
I have a case open (43590888), have attached numerous comments and packet captures and no one's reviewed anything. Ticket status is "Waiting for Support". I've called in 2x, first time I waited on hold for 18 minutes, the Tech didn't solve the problem after maybe 2 hours. 2nd time I called in, I got through in about 5 minutes, spent another 2 hours, Tech didn't solve the problem. Tried calling a 3rd time this evening, waited on hold for 20+ minutes, then the system hung up on me (I have an Android phone).
How do I get the hardware swapped out? I've spent about 30 hours of my time on this, and there's no way I can bill my client for it. I need this resolved, or I'm going to have to yank out the Sonicwall, and then try to get a refund from my vendor. Sad, this is the worst problem I've ever had in all the years I've been using Sonicwalls.
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
I have no idea where the l2 mcast errors could be originating at - especially when the Sonicwall is claiming that 192.168.0.77 is a multicast address, and that the destination of 107.125.50.51 is somehow a problem. UDP packet, port 4500. I suspect that these are the packets which are reauthorizing the phones with IKE keys so they can use wifi calling.
Tried rebooting the iphones, have IGMP and MLD snooping configured on all the switches throughout the property.
I have a case open (43590888), have attached numerous comments and packet captures and no one's reviewed anything. Ticket status is "Waiting for Support". I've called in 2x, first time I waited on hold for 18 minutes, the Tech didn't solve the problem after maybe 2 hours. 2nd time I called in, I got through in about 5 minutes, spent another 2 hours, Tech didn't solve the problem. Tried calling a 3rd time this evening, waited on hold for 20+ minutes, then the system hung up on me (I have an Android phone).
How do I get the hardware swapped out? I've spent about 30 hours of my time on this, and there's no way I can bill my client for it.
Thanks for the tip Karan, we tried what you suggested long before I posted here. Sometimes the calls go out, sometimes not - when they don't, then perhaps 30 seconds later they'll be able to make an outbound call.
Usually inbound calls don't work, sometimes they do. Packets inbound on X1 for the X1 IP address get dropped. Doesn't matter if I have a rule + NAT, the NAT never gets hit.
Sometimes the phones on the LAN try to go out to the Wifi calling server on the internet, and that packet gets dropped with the weird l2 mcast but unicast dest (#1) error message is logged in the packet capture.
I had a spare TZ600 unit in my office that I configured the same as my client's - connected it to my cable modem (also comcast), and it worked flawlessly. Inbound/outbound calls - no problem.
Went thru my client's TZ600 config, and the only difference was in the Diag menu involving the randomization of TCP sequence #'s - it was enabled on my client, not in my spare unit. Disabled it on the client side - didn't make a difference.
Tried downgrading my client's TZ600 firmware to the same rev as what I had on the office TZ600 - no difference.
I have no idea where the l2 mcast errors could be originating at - especially when the Sonicwall is claiming that 192.168.0.77 is a multicast address, and that the destination of 107.125.50.51 is somehow a problem. UDP packet, port 4500. I suspect that these are the packets which are reauthorizing the phones with IKE keys so they can use wifi calling.
Tried rebooting the iphones, have IGMP and MLD snooping configured on all the switches throughout the property.
I have a case open (43590888), have attached numerous comments and packet captures and no one's reviewed anything. Ticket status is "Waiting for Support". I've called in 2x, first time I waited on hold for 18 minutes, the Tech didn't solve the problem after maybe 2 hours. 2nd time I called in, I got through in about 5 minutes, spent another 2 hours, Tech didn't solve the problem. Tried calling a 3rd time this evening, waited on hold for 20+ minutes, then the system hung up on me (I have an Android phone).
Did you find a solution to this? We have a similar issue with our AT&T wifi calling.
Sadly, no. Never got a solution to it. Ended up getting Sonicwall to swap out the TZ600 w/new hardware. Completely reconfigured the device, still had the problems. Replaced all the Comcast cable connectors in the home - minor improvement on speed when we yanked out an old coax cable no one knew was there, but no effect on the drops with Wifi calling.
Ripped out the replacement hardware - it's sitting on the floor of my office because it was way past the date I could return it to the vendor. Built the customer a firewall using pfSense - still had issues with wifi calling. Adjusted the UDP timeouts, ran some continual ping tests to the AT&T wifi calling server IP's from the customer, and from my office - then compared and contrasted the ping times, latency, and hops. They are in the same city as my office, but different neighborhood nodes. They have residential service, office has commercial.
Office has lower latency and less hops than the residential. Obviously taking two different paths - but not sure if that was the cause of it or not.
pfSense firewall was problematic, so I ripped that out and rebuilt using Ubuntu 20.04 and configured it exactly as I have in the office. Works somewhat better, but Customer still complains about call drops at random... no real rhyme or reason about it. We looked at weather-related issues, underground cable, pole cable, neighborhood node performance (there's 92 other homes on the node and none of them are complaining)...
I have zero issues when I test it with my Android phone. I suspected her phone, but she's upgraded and gotten a new one - still has the issues. Really, really, really frustrating to have to deal with error messages that no one could decode or figure out, and have it impact wifi calling at this period of time.
Interestingly enough, I don't have this problem with any other customers that have virtually the same equipment in the neighborhood and who share that node, or who use Sonicwall products.
I'm about ready to buy a box of Kosher salt and pour a circle of protection around the house...