2.4Ghz high latency solutions

  • 0
  • 1
  • Problem
  • Updated 2 years ago
  • Solved
I'm currently experiencing some issues with our wireless network I could use some help with figuring out a solution.

The situation is as follows:

I've two EW C25s and 60 AP-3825is. About every classroom, office and common area has an AP. I've the following configured:

The main wlan SSID (2.4+5ghz) is bridged at the AP and connects to internal resources.
The guest network (2.4+5ghz) is a captive portal bridged at the C25 and forwarded to the gateway.
The BYOT network (2.4+5ghz) is RADIUS auth bridged at the C25 and forwarded to the gateway.
The Chromebook network (5ghz) is bridged at the C25 and forwarded to the gateway.

For the most part this work out fine, but recently I noticed a massive increase in latency on the 2.4ghz main network. This is causing problems with new teacher laptops (2.4ghz b/g/n only adapters in them I'm afraid). Pings between 500-2000ms, and terrible upload speeds. It's causing severe disruptions.

What are some things I should check? I have a feeling this is part configuration and part hardware incompatibility. Does anyone recognize this situation?
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb

Posted 2 years ago

  • 0
  • 1
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
Do a packet capture and check for cochannel interference.   What is your ap density? 
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
I've never done one before. What would you recommend?

Density is one per classroom, some are in closer proximity than others (room sizes differ a lot). I don't really know how else to describe it.
Photo of JP

JP

  • 984 Points 500 badge 2x thumb
How are the channels and power set on the 2.4 radio's ? Auto channel and max power ?  Depending on your floor plan you might have a lot of channel interference in 2.4.  Have you used some type of scanner to see how many AP's are on the same channel in a given area ?  You can use something as simple as the airport utility on an iPhone as a starting point.  2.4 is also more susceptible to outside interference sources. 
(Edited)
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
Auto channel and max 14 dBm/8dBm 2.4/5. Looks like most are on 1, 6, 11; 36 and 44.

There's one rogue AP that's part of a point to point deployment that's somehow broadcasting.
(Edited)
Photo of Joshua Puusep

Joshua Puusep

  • 2,274 Points 2k badge 2x thumb
If you have auto Tx control enabled, you should verify that you have the RF domain setup appropriately on each AP.  Typically you would have one RF domain for 5.0Ghz and one for 2.4Ghz in each building, which means that all of the AP's in the building would send out probes and adjust their signal strength based on the closest neighboring AP's.  If the neighboring AP's are not in the same RF domain, they will most likely default to the max allowed strength.  This could cause a whole lot of channel overlap in the environment you described.  You could also manually set the channel's and power, however you would need a good tool to do this correctly, something like Ekahau site survey or Fluke Airmagnet.

You can also look at your DCS settings and Ap logs as well to determine which AP's are getting slammed by interference.
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
Where are these RF domains defined? They seems to be on default MyDomain for both radios on all APs.
(Edited)
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 47,434 Points 20k badge 2x thumb
Troubleshooting 101 = isolate the problem

- does it happen 24/7 or only during peak hours
- how many clients are connected to the system and/or per AP
- software version
- what did you ping ?

"For the most part this work out fine, but recently I noticed a massive increase in latency on the 2.4ghz main network."
So it happen only in the SSID main and not on the other ones ? That sounds weird and doesn't indicate a issue with channel interference.

There are not enough information to give you any advice at this point beside that you'd need to isolate it more to search in the right direction.
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
This is a constant occurrence.

There's a lot of APs with 0 clients, but some will go up to 50 depending on Chromebook usage. Occupancy doesn't appear to matter.

Software version is 09.15.07.0008

I ping the local gateway.

Response times are a little on the high side, but with this particular laptop model (HP ProBook G3 455 / RTL8723BE) it's more severe. I disabled all APs/SSIDS save for the one I need these to work on, strict-n:

I connect my Samsung S5. Ping to the gateway gives min 1ms \ max 89ms \ avg 12.6ms

I connect my iPad Air. Ping to the gateway gives min 1.7ms \ max 40.3ms \ avg 26.7ms

I connect the HP ProBook to the gateway gives min 101ms \ max 2967ms \ avg 410ms

However. when I force the AP in b/g mode the results are much better: min 2ms \ max 3ms \ avg 2ms

If anything it seems a driver problem, but I don't know if it's a setting on the machine or the AP I need to change to make them play nice together.

I'm going to try a few driver versions and see if it helps.
Photo of James A

James A, Embassador

  • 6,762 Points 5k badge 2x thumb
09.15 is pretty ancient, I would try 09.21.12.

Apart from the 2.4GHz only devices, what's the 2.4/5GHz split amongst clients? You could consider removing the other SSIDs from some 2.4GHz radios to give some breathing room for your HP ProBooks.

Definitely worth doing a RealCapture to see where the packet delay is occurring too.
Photo of Jeff White

Jeff White

  • 142 Points 100 badge 2x thumb
If you have an AP in every classroom, office and common area, I would turn off auto tx and statically set the 2.4ghz radio to no more than 5dBm. This is still more power than you need on each AP. I would actually recommend disabling at least half of your 2.4ghz radios but without doing a proper site survey and not knowing building materials, no one would be able to tell you how many to disable, which ones to disable and what power to turn the remaining radios up to. 

The issue very well may be a co-channel contention issue. This will eat up airtime on the AP and cause high latency on the network. Because every channel is a contention domain, if an AP can hear another device on the same channel, it can not talk and must wait for the channel to be clear before it can send data. This causes data to queue at the AP and cause latency. 

I would start with updating firmware because they have had a lot of buggy versions for what seems like 2 years now. Then I would get a print out of your floor plan and write down the channels the APs are on and adjust if needed. Co-channel contention is unavoidable on the 2.4ghz band, but if you plan appropriately, yo can minimize the effect it has on the network. After that, assuming you have one in every classroom, start turning them down to 5dBm.
Photo of Jeff White

Jeff White

  • 142 Points 100 badge 2x thumb
Another thing to think about would be to add USB wireless adapters to the 2.4ghz only devices. Since the devices are teacher devices, one would hope they wouldn't continually go missing. Just remember an 802.11ac adapter will need a USB 3 port to get 802.11ac speeds. 

https://www.amazon.com/dp/B00WT80LPK/ref=twister_B01EMS6TVE?_encoding=UTF8&psc=1
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
I inherited this network and the NetSight server install is not working properly. Thus far I've been unable to update the firmware on anything Extreme (inventory manager can't find any new firmware and I can't download them manual from extreme). It's on my to do list.

Adding usb wireless adapters isn't an option. I and the teachers are pretty unhappy with the current amount of stuff they already need to plug in (HDMI/USB to touch board/USB to document camera/flashdrives etc). I've run plain Cat-5 to every laptop until I can resolve the wireless issues, yet another thing to plug in. I'd much rather blanket replace the internal mini PCI-E bgn realteks for intel AC adapters. I'm going to try and swap the chip from a derelict I have laying around.

I wasn't part of the organization when purchasing decision for the laptops were made (though I wish I was), I just need to deal with the implementation now.
(Edited)
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 47,434 Points 20k badge 2x thumb
"I connect my Samsung S5. Ping to the gateway gives min 1ms \ max 89ms \ avg 12.6ms"
"I connect my iPad Air. Ping to the gateway gives min 1.7ms \ max 40.3ms \ avg 26.7ms"
"I connect the HP ProBook to the gateway gives min 101ms \ max 2967ms \ avg 410ms"

Could you please check the clients report - I'm interested what the "protocol" for every model is (on a AP that is set set to b/g/n or g/n - whatever you've configured).

Also check the > VNS > global > wireless QoS > FCA > set it to 100% airtime.
And set the > VNS > WLAN services > main > QoS > set it to WMM & 802.11e (disable the other options) and enable Flexible Client Access.

If you still have the high delay set the test AP to the following settings... in short disable all the fency stuff..



You'd also replace the WLAN adapter - I've done that with my laptop NIC and it took me only 5minutes and it's very cheap.
https://www.amazon.de/Intel-7265%C2%A07265%C2%A0NGW-802-11%C2%A0ac-Bluetooth-4-0%C2%A0867%C2%A0MBPS/...

That would enable the laptops for 802.11ac so you'd use the 5GHz band with a much higher bandwidth.
Photo of Jeff White

Jeff White

  • 142 Points 100 badge 2x thumb
Ronald, do you not notice lower throughput on your wireless network with ADDBA and A-MPDU disabled? That's essentially taking away the major improvements in 802.11n standard. I'm assuming you did that for simplicity sake back in the days of the bad firmware and the little yellow ! of doom. 
(Edited)
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 47,434 Points 20k badge 2x thumb
I've it disabled because there are far too many issues with it in my opinion - either bug related or because the WLAN adapter of the clients can't deal with it and behaves in a strange way.

http://www.extremenetworks.com/search/?_ga=1.113200580.1562969531.1460041509#q=addba&t=All&s...

I know I'll loose some functionality but I prefer a working network even that result in a minor waste of bandwidth :-)
Photo of Jeff White

Jeff White

  • 142 Points 100 badge 2x thumb
Makes sense to me. Thanks!
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
I will give these settings a try when the environment isn't in use (after school's out). I'm also expecting a call from level 2 HP support to troubleshoot, I highly suspect it's the wireless adapters having a driver problem with 2.4n. It may be one of the settings you recommend changing.
Photo of Diederik Kuijper

Diederik Kuijper

  • 334 Points 250 badge 2x thumb
I gave GTAC a call and we've updated the controllers and APs to 09.21.12.005. This resolved the issue partly.

Latency still tanks occasionally, but not as bad.