Question

Problems with old WLAN Clients after changing the WLAN infrastructure

F

Hello,

 

we switched our wlan infrastructure at one of our warehouses from old Motorola Symbol Aps using a WEP SSID to our new wlan infrastructure using the XIQ Cloud, with primarily AP410 and a PPSK SSID, using one PPSK for every forklift in the warehouse. (They use an old Win7 embedded OS).

 

After the switch we noticed several disconnects within the Logistics Software we are using on the terminals to manage the warehouse. We Software keeps losing the connection to the server for a few seconds (from 2-3 seconds to 15-20) and then reconnects. It seems like it’s happening all over the warehouse, even at locations where i can see with our other devices that the wlan connections seems good and stable. What i’ve noticed is that it looks like some sort of roaming problem to me. It seems to me like it is some sort of roaming problem. Maybe the terminals take too long to roam between the aps or having problems with PPSK authentication?

 

I checked the client monitor to check the clients and noticed a poor client health. Seeing many issues displayed as “The client is configured with an incorrect static IP address or gateway.” Which in fact, seems not to be correct. The forklifts are configured with a static ip, dns and gateway (we dont use DHCP in this VLAN). So what is the cause of this error? Could the be also a hint to our main wlan problem? 

 

If you need any more information, dont hesitate to ask. I appreciate every help. :)

 

31 replies

F

Hello Orvais,

 

we will contact our serice partner next week and let them do a proper analysis of the situation and hopefuly get to fix this.

 

Something i noticed today after i installed v10.3.1 and disabled Weak SNR Supression / Safety Net, the aps seemed to choose their channels pretty bad:

I instantly got a high amount of Channel Interference messages in XIQ. But why are they doing this? I thought there would be some sort of automation which prevends such bad channel selection. Or is this the “normal case” after doing a reboot on all aps? Never noticed this before. 

 

Edit: never mind. They now set their channels much better. Guess they need their time to check neighbors, etc.

O
Rank
Userlevel 5

Hi Florian,

Water blocks radio waves in general; it happens to be particularly good at absorbing radio waves near 2.4 GHz frequency. If you have shelves full of water bottles/boxes, they would cause signal loss but NOT Co-Channel Interference (CCI).

In Co-Channel Interference, APs on the same channel will hear each other and defer. For example, if AP-1 on channel 6 hears the preamble transmission of a nearby AP-2, also transmitting on channel 6, AP-1 will defer and cannot transmit at the same time. Likewise, all the clients associated to AP-1 must also defer transmission if they hear the preamble transmission of AP-2. All these deferrals create medium contention overhead and consume valuable airtime because you have two basic service sets on the same channel that can hear each other.

 

Co-Channel interfence can’t be completely avoided but it can be managed with a proper WiFi design.

1- In some cases, you can move SSIDs away from 2.4GHz to 5GHz, 5GHz will provide a lot more channels and there will be a lot less chance of CCI.    

2- In other cases, turning off 2.4GHz radios on some APs help. If the coverage is good, turning some 2.4GHz radios off on the APs will help reduce CCI. Before this is done, a proper site survey should be conducted to detrmine which 2.4GHz radios can be turned off to avoid converage holes.  

3- Another major cause of CCI is the WiFI clients themeselves, more importantly the ones always on the move, but this can’t be avoided in your environment.  

I would recommend you to contact the system integrator and ask them to perform a WiFi survey to determine the best way of reducing CCI in your network. They might need to change some AP placements/reduce the no. of 2.4GHz radios/implement a better channel plan etc. But there is a way to manage the CCI.

Let us know if you need further assistance.

Regards,

Ovais    

F

Hello Orvais,

could the full water boxes alsocause the interferences? 

 

And i am only using WLAN VLANs on the Switch Ports. We segmented our client vlans pretty broad to get every client department covered (logistics, production, IT, office, etc.). So we got around 35 client VLANs. Currently on the switch port there are 39 VLANs configured. We got this configured on all our plants and the others do not seem to cause so much trouble (Hivemanager Classic).

 

I will try to get Weak SNR and Safety Net disabled. Could I try something else to get this solved? 

 

Thanks in advance. :)

 

O
Rank
Userlevel 5

Hi Florian,

The tech dump shows data that points to CCI, you have to address co-channel interference in the network. See following.


2021-03-22 13:31:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 13:25:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 13:00:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 12:36:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 11:01:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 10:46:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 10:24:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 10:19:48 err     ah_dcd: wifi1: A high interference alert was raised.
2021-03-22 09:37:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 09:06:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 08:56:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 08:34:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 08:27:48 err     ah_dcd: wifi1: A high interference alert was raised.
2021-03-22 08:25:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 08:09:48 err     ah_dcd: wifi1: A high interference alert was raised.
2021-03-22 08:06:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 07:57:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 07:40:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 07:34:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 07:01:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 06:52:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 06:20:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 06:12:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 05:52:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 05:13:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 04:59:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 02:28:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 02:11:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 02:00:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 01:50:48 err     ah_dcd: wifi0: A high interference alert was raised.
2021-03-22 01:44:48 err     ah_dcd: wifi0: A high interference alert was raised.

Remove non-wireless VLANs off the APs switch ports, you've got an mDNS storm being spread to the wireless network from the wired network.

In addition, you can now also disable Weak SNR suppression and Safety Net in the radio profiles.

 

Regards,

Ovais

 

 

F

Hello Orvais,

 

thanks for your response. The Problem here is that it’s difficult from my point of view to make changes to the main SSID (PPSK). The employees there are already exasperated by the situation, if changes make it worse, it grows. I think you know what i mean.

 

I disabled 5 Ghz in the SSID settings of the policy and checked it via show int. The ssid is not broadcasted on 5 Ghz. I did not make any changes to the other SSIDs:

I see that many of our mobile devices are also in 2.4 Ghz. But they are spread across the plant.

So the only way to decrease the utilization would be to make room in the 2,4 Ghz for the terminals? I could try to disable it on MDM and 802.1x - but that would probably cause some issues with the mobile devices and notebooks at the plant.  

I will send you the tech data to the e-mail. I am on holidays next week so dont wonder if i i am not responding.

 

Thanks so much again for your assistance!

O
Rank
Userlevel 5

Hi Florian,

Here is my response to your questions. 

so i got news from the employees of the plant today. I configured two of the forklifts into the “stapler” ssid with 5 Ghz turned off and set all the settings you suggested. It seems like one of the forklifts isn’t able to be used now anymore (worked after i configured it to the ssid and waited about 20 mins) and the other one is also experiencing way more disconnects than before. I am trying to get some better answers from the employees.

(Ovais) When you enabled the Staples SSID on 2.4GHz, did you either disable or moved other SSIDs to 5GHz at that time? if not, then based on the utilization stats that I mentioned in my previous post, enabling Staples SSID on 2.4GHz while other SSIDs were also broadcasted in 2.4GHz band won’t make any difference. We want to reduce the utilization on 2.4GHz, and the only way to do it is to reduce th eno. of SSIDs on it, because you already have 16 APs per channel in 2.4GHz spectrum. 

Is this somehow bugged or do i missunderstand something in this view? Why did it not use any of the 2,4 Ghz channels?

(Ovais) From the screenshots, looks like the clients were still connecting to 5GHz. Make sure when you disabled SSID on the 5GHz radio and updated the configuration on the APs, they actually took it. You can verify that by selecting an AP Actions>Advanced>CLI Access then use the command “show interface”, this will show you the SSID mapping on the AP radios. Use it to verify that APs are still not broadcasting Staples SSID in 5GHz, if they are, then you probably need to push a “Full Configuration update” instead of “Delta update”, which will cause APs to reboot. 

 

What happened exactly in the red marked and other times? It seems the client is connected to the ap but doesnt show any dbm or usage. I got these on every terminal throughout the day. And of course the error messages about “wron static ip or default gateway”

(Ovais) This to me looks like the same utilization issue that I had explained earlier, the same channel is being used by many APs and the mgmt. overhead traffic increases so much that the clients do not get a transmit opportunity. This was seen in the .pcap files you shared as well.  

 

There’s also a really high amount of “station is deauthenticated from thru SSID” messages. How do i interpret them correctly?

 

One of the causes could be interference, but I'd like to get detailed messages on the rejection. Could you SSH into the AP you are testing on and run the following commands?

_debug auth basic

_debug auth verbose

Then try to connect to this AP a couple more times to replicate the issue. Once that is done, please collect tech data and send that to me. Feel free to email it directly to me at mqayyum@extremenetworks.com.

To collect tech data, go to Tools> Utilities> Get tech data> Check the box next to the device> Get tech data (blue button at the top of the page this time).

 

Regards,

Ovais  

F

There’s also a really high amount of “station is deauthenticated from thru SSID” messages. How do i interpret them correctly?
 

 

F

Another thing i just noticed are these logs from the clients

What happened exactly in the red marked and other times? It seems the client is connected to the ap but doesnt show any dbm or usage. I got these on every terminal throughout the day. And of course the error messages about “wron static ip or default gateway”

F

Hello Orvais,

 

so i got news from the employees of the plant today. I configured two of the forklifts into the “stapler” ssid with 5 Ghz turned off and set all the settings you suggested. It seems like one of the forklifts isn’t able to be used now anymore (worked after i configured it to the ssid and waited about 20 mins) and the other one is also experiencing way more disconnects than before. I am trying to get some better answers from the employees.

 

I checked the channel capabilites of the terminal.

Is this somehow bugged or do i missunderstand something in this view? Why did it not use any of the 2,4 Ghz channels?

The other one which is still somehow working shows some different values.

 

P.S.: But i could check if disconnects happen when the forklifts stands still and is permanently connected to an ap. This was stable so far (also for about 20mins).

O
Rank
Userlevel 5

Ouch ….!! this is new :)

Yes, if you broadcast the same SSID on both bands and the clients are highly mobile, there is a good potential that you will see the clients jumping between the bands all the time. Mainly because you have different coverage overlaps for 2.4 and 5GHz bands (much smaller for 5GHz) and with Band Steering enabled it can make things difficult for roaming clients.    

If you can also get rid of SSIDs that are not used or can be consolidated by using “user profiles” in the configuration, it will greatly help you with the WLAN performance, because you have only three channels and a lot of APs causing too much CCI, having more SSIDs will increase mgmt. overhead traffic on the channel by quite a lot which means less airtime for clients. Based on the following rough overhead calculation, if you have 16 APs on the same channel (50APs/3 channels= 16 APs) with 4 SSIDs, the mgmt. overhead is almost 100%, which means almost no airtime for clients to send data. 

16 1 SSID= 37.84% 2 SSIDs= 75.69% 3 SSIDs=100.00%

4 SSIDs= 100.00%

     

In comparison to 2.4GHz, it's much better in 5GHz because you have a lot more channels to work with (Channels UNII-I 36,40,44,48 and UNII-III 149 to165). with the same 4 SSIDs the mgmt. overhead on the 5GHz network is only 26.69%.

16 1 SSID = 6.67% 2 SSID = 13.34% 3 SSIDs = 20.02% 4 SSIDs= 26.69%

 

If you think all clients can support 5GHz and 5GHz coverage is good, you should move this SSID to 5GHz. If not all the clients support 5GHz OR the 5GHz coverage is not uniform then only keep this SSID on 2.4GHz  (with 1 SSIDs and 16 APs sharing the channel the overhead is around 38%) and move the rest to 5GHz.

First, with the current PPSK SSID try broadcasting the SSID only in 2.4GHz or 5GHz (make sure you are using all non-DFS channels in 5GHz) based on what the clients support, monitor it, if it works, good deal. Otherwise, try the changes I mentioned in my previous post.

 

Regards,

Ovais

F

One more thing I’ve noticed yesterday. I saw some forklifts for a short time in 5 Ghz channels and so i checked the networks cards of the terminals again and noticed that they are indeed capable of 802.11a.

Could this also be causing some sort of trouble? Wifi1 is configured to 802.11ax

F

Hello Orvais,

 

i created the SSID Stapler with WEP to check if WPA2 was causing some of the problems we are facing. (Our old Infrastructure used WEP) and put only one of the forklifts in it. I guess this is also the reason why it the SSID consumes so much of the Airtime?

I was planing to change the SSID back to WPA2 PPSK anyway next week when i am on the plant. If i remember correctly, WPA2 and PPSK should make no difference in terms of authentication time, etc.? And i will also make every changes to the SSID like you suggested.

Here is the #show from the main AP in the loading hall:

#show int wifi0 _count
694331840 rx frames
28185661 rx data frames
        27797700 rx unicast data frames
        315217 rx multicast data frames
        72744 rx broadcast data frames
314482305 rx management frames
14698 rx control frames
0 rx unknown frames
14511 rx BAR (Block Ack Request) frames
694331840 11n rx frames
0 rx chip keyix errors
0 rx chip antenna errors
0 rx chip short frame errors
0 rx ieee queue depth
1363323 rx Retries
4% rx retry rate
103895730 rx CRC errors
13% rx CRC rate
49607 rx frame errors other than CRC
0 hw FIFO overrun
3568 decryption failed
0 MIC failure
194 rx frames dropped
        0 frame too short
        194 frame too large
        0 frame in other mode
0 rx dropped because no skbuff
40450324 tx data frames
        34967300 tx unicast data frames
                34967300 tx SU unicast data frames
                0 tx MU unicast data frames
        3955912 tx multicast data frames
        1527112 tx broadcast data frames
        --------------------------------
        40119242 tx WMM best effort data frames
                0 tx WMM best effort MU data frames
        54772 tx WMM background data frames
                0 tx WMM background MU data frames
        276310 tx WMM video data frames
                0 tx WMM video MU data frames
        0 tx WMM voice data frames
                0 tx WMM voice MU data frames
19961675 tx management frames other than beacon
166373135 tx beacon frames
0 tx beacon frame stucks
0 PCI errors
4504972 tx BAR (Block Ack Request) frames
21394174 tx aggregated completions
0 tx BAR (Block Ack Request) frames dropped
0 invalid tx rate index
0 invalid rx rate index
77947 tx retries
        56281 tx RTS failures
        21666 tx retries
0% tx retry rate
0% unicast data tx retry rate
0 tx retries of sub frames
6230461 tx frames with no ack marked
1544020 tx frames with rts enabled
0 tx frames with cts enabled
1428498 tx frames with short preamble
145945 tx frames with an alternate rate
0 tx frames with protection
34713 tx frames dropped
        3442 no tx buffer (data)
        31271 no tx buffer (mgmt)
4979728 tx frame errors
        4979728 too many hw retries
        0 hw FIFO underrun
        0 transmit filtered by hw
0 tx transmit queue full
1024 tx available buffers
0 exceeded txop
0 exceeded tx timer
10997867 tx frames succeed
5822143 tx frames failed
17 rx rssi from histogram
0 no skbuff available for beacon
0 switched from HT40 to HT20
0 switched from HT20 to HT40
1094489 channel switches
0 AMRP one-way neighbor timeout detected
1390 interference raise alert
1389 interference clear alert
0 band steering suppress
0 load balance suppress
347199 weak snr suppress
188161 safety net bypassed suppress
 

Another one within the halls:
#show int wifi0 _count
980976677 rx frames
3901107 rx data frames
        3761585 rx unicast data frames
        122019 rx multicast data frames
        17503 rx broadcast data frames
570644463 rx management frames
1877 rx control frames
0 rx unknown frames
1877 rx BAR (Block Ack Request) frames
980976677 11n rx frames
0 rx chip keyix errors
0 rx chip antenna errors
0 rx chip short frame errors
0 rx ieee queue depth
400458 rx Retries
10% rx retry rate
73190042 rx CRC errors
7% rx CRC rate
29554 rx frame errors other than CRC
0 hw FIFO overrun
1889 decryption failed
0 MIC failure
4 rx frames dropped
        0 frame too short
        4 frame too large
        0 frame in other mode
0 rx dropped because no skbuff
4142921 tx data frames
        2682117 tx unicast data frames
                2682117 tx SU unicast data frames
                0 tx MU unicast data frames
        984215 tx multicast data frames
        476589 tx broadcast data frames
        --------------------------------
        4046906 tx WMM best effort data frames
                0 tx WMM best effort MU data frames
        30958 tx WMM background data frames
                0 tx WMM background MU data frames
        65057 tx WMM video data frames
                0 tx WMM video MU data frames
        0 tx WMM voice data frames
                0 tx WMM voice MU data frames
9674441 tx management frames other than beacon
166265508 tx beacon frames
0 tx beacon frame stucks
0 PCI errors
1210803 tx BAR (Block Ack Request) frames
2209963 tx aggregated completions
0 tx BAR (Block Ack Request) frames dropped
0 invalid tx rate index
0 invalid rx rate index
33732 tx retries
        24795 tx RTS failures
        8937 tx retries
0% tx retry rate
1% unicast data tx retry rate
0 tx retries of sub frames
2401268 tx frames with no ack marked
604116 tx frames with rts enabled
0 tx frames with cts enabled
373706 tx frames with short preamble
69554 tx frames with an alternate rate
0 tx frames with protection
159 tx frames dropped
        0 no tx buffer (data)
        159 no tx buffer (mgmt)
1376147 tx frame errors
        1376147 too many hw retries
        0 hw FIFO underrun
        0 transmit filtered by hw
0 tx transmit queue full
1024 tx available buffers
0 exceeded txop
0 exceeded tx timer
3147329 tx frames succeed
9198734 tx frames failed
25 rx rssi from histogram
0 no skbuff available for beacon
0 switched from HT40 to HT20
0 switched from HT20 to HT40
1231140 channel switches
0 AMRP one-way neighbor timeout detected
533 interference raise alert
533 interference clear alert
0 band steering suppress
0 load balance suppress
411130 weak snr suppress
156358 safety net bypassed suppress
 

O
Rank
Userlevel 5

It took me some time to check the packet capture, only one capture file had some meaningful data. The good thing is at least in the captured data the RF looks ok. I don’t see any retransmission of the frames and if you see the packet sequence, there is not much of a difference RSSI. it looks consistent.

One other thing that I have noticed is the channel utilization caused by the SSID “Stapler”, this SSID is comsuming most of the airtime on channel 11. I could only see channel 11 in the capture because the AP you might have used to capture the data was on channel 11 as well.   

The Block Ack that you mentioned in your previous response is not a concern, its just a mechanism for the AP to send acknowldements in blocks instead of individual acknowledgments which consumes more airtime. 

  I say you try following settings in the AP device template under radio interface settings:

 

 

Deny connection requests from legacy clients using - 11b (Enable to get rid of 11b legacy clients)

Background Scan - OFF

High Density Configuration - OFF

Band Steering - OFF

Client Load Balancing - OFF

Tx Beamforming - OFF

MU MIMO - OFF

Dynamic Channel Switching - OFF (keep it OFF for a while)

Weak Signal Probe Request Suppression - ON (set value to 15dbm first and then wait to see the diff., it should help free up the channels)

If all of this does not improve the situation then I would recommend to create a test WPA2 SSID and connect a test client with it to test the connectivity and see if faces the same problem.  

Send me output of #show int wifi0 counter from few APs where you see issues the most. 

 

Regards,

Ovais 

F

Hello Orvais,

 

if i remember correctly it was a some sort of redundancy coverage. Some areas had redundancy some not. But i am not shure and none of my colleagues is. 

 

One feedback i got yesterday from our plant is that in the loading street where we have a pretty empty hall, they also get many disconnects within the application. They also said the more forklifts in the loading hall the more disconnects. They even said that it looks like that only one forklift is able to work and then the next one “in the row” gets a connection and is able to work.

 

Most of the things i posted in here were from the “main” ap in the middle of the loading hall. It’s this area on the screenshot i posted, with three APs (channel 1,6 and 11). The ones you can see at the top or in the corners are either outdoor aps or in different halls.

I have checked the three aps in the hall right now and their summary state is on all three “good”. The CRC rate right now is between 8% and 17%. I will check these three aps again as soon as the number of forklifts in the loading hall rises up due to more trucks being loaded.

 

Could it help to configure everything down have the most sort of default wlan, atleast for the 2,4ghz? Like deactivating every sort of helping mechanism (short guard interval, band steering, etc.), activating every bitrate, raise the max. signal strength, since we only need the forklifts to be working properly.

Like i said, i am sure the old infrasctructure had massive overlaps in channels and cells. 

 

And no, sadly i dont know these antenna kits and couldn’t find them on the web so far. 

 

Can you check if this link works? I may have to creater longer captures:

https://www.dropbox.com/s/11vo1zmd8s3aie6/local-2021031575615UTC-BCF3103F6680-wifi0.pcap?dl=0

https://www.dropbox.com/s/ktl9mcvhgdz9dwp/local-20210312124219UTC-BCF3103F6680-wifi0.pcap?dl=0

 

O
Rank
Userlevel 5

Hi Florian,

You can upload the capture files to OneDrive/SharePoint or any other file-sharing service and share the link here. We have some obvious variables, 67 Vs 50 APs, Internal Vs external antennas which will change the coverage in your environment 100%. As for your question on “shouldn’t the newer and more “intelligent” aps able to cover the area as good as the old ones or at least more optimized?”   

YES, provided we stick by the fundamentals of RF design, just can't beat the physics. And, let's also consider the fact that you are still using the new APs with the same old clients, which will make your APs work in backward compatibility mode. Hence, performance will be limited to the capability of clients, you won’t be able to take full advantage of the advanced RF features that the new AP has to offer.   

Do you remember which external antennas on the AP5131 did you use? model no. will help. I can then compare the coverage pattern on AP5131 external with external and the AP410C.

 

Regards,

Ovais

F

Hello Orvais,

 

I just checked the CRC Rates on two of the Aps which had a high rate on Friday again and know it’s pretty low. One ist at 9%, the other at 13%. The rretry rate is at 4% and 5%.
 

I’ve got two packet captures (one from friday and one from today). How i should i post them here? Guess with the character limit of 3000 i cant post really much. 

What I’ve noticed so far is a decent amount of blocked requests by the ap because of too low bitrates (basic is 11mbit most of them are either 1mbit or 9).

 

 

But shouldn’t the newer and more “intelligent” aps able to cover the area as good as the old ones or atleast more optimized? We did place them like the wlan coverage by our partner reccomended them. I would guess that the motorola aps had way more overlapping cells than the current one causing more interferences. But that is only guess. I never heard of any intense problems with the old wlan for as long as i am working in the company.

 

We were not told about any disadvantages of using the 410cs for our infrasctructure.

 

 

O
Rank
Userlevel 5

I would wait for packet capture (you can take it remotely from any AP by following the screenshot in my previous response) to see if my suspicion about retransmissions is correct, which in fact will cause the application to terminate the session. If we see retransmission of packets then it would solely be an RF design problem, Tx power wouldn’t play much of a role in there.

Looking at the CRC error rate alone i.e. 46%, not many latency-sensitive applications will survive that.

Some other clues from our discussion so far:

1- 67 APs were replaced with 50 APs.

2- External antenna Vs. integrated antennas (this one alone can be causing this entire issue).

3- External antennas may have had different gain and radiation patterns for sure, which doesn’t match your current integrated antenna APs.

4- If the APs are deployed at a height of 30-36 feet (normal for warehouses) then integrated antennas are not a good choice due to the dipole type side-side Omni radiation pattern, which won't focus most of the signal on the ground.   

 

Regards,

Ovais 

 

F

Yes. They are already configured like your screenshot. 

D
Userlevel 2
Badge +1

13-15 is what extreme says in their workshops.

 

Did you check your Rate Settings?:

 

F

Ah, missed that one. Thanks. :)

 

I am currently at max DB 14. Do you have any recommendations on how low i should minimum go? This already seems to be pretty low. 

D
Userlevel 2
Badge +1

Try what i told you some posts ago :)

F

So this one is from our “main ap” in the truck loading street. This is a very long hall with three aps in it. Its pretty empty besides the trucks in the middle of it getting loaded. 

 

AC=access category; be=best-effort; bg=background; vi=video; vo=voice;
AIFS=Arbitration Inter-Frame Space; Txoplimit=transmission opportunity limit;
IDP=Intrusion detection and prevention; BGSCAN=background scan; PS=Power save;
HT=High throughput; A-MPDU=Aggregate MAC protocol data unit;
DFS=Dynamic Frequency Selection; CU=Channel Utilization;
EIRP=Effective Isotropic Radiated Power (Transmit Power + Max Antenna Gain + Max                                                                                                                                                              TX Chains Gain);
OFDMA=Orthogonal Frequency Division Multiple Access;
BSS_Color= Basic Service Set Color; TWT=Target Wake Time;

Summary state=Fair;
Mode=access; Radio disabled=no;
Admin state=enabled; Operational state=up;
MAC addr=bcf3:103f:6690; MTU=1500;
Freq(Chan)=2462Mhz(11); EIRP power=17.01dBm(10dBm + 4.00dBi + 3.01dBi); Diversit                                                                                                                                                             y=disabled;
Tx range=300m; Noise floor=-96dBm; Tx power control=disabled;
Radio profile=default_2.4ghz_gn; IDP profile=N/A;
Application Essentials profile=N/A;
Presence profile=N/A; Presence state=disabled;
Beacon interval=100; Max clients number=100;
Phymode=11ng; Short preamble mode=enabled;
Tx Chain=static 2; Rx Chain=static 2;
A-MPDU=enabled; Short guard interval=enabled;
Channel width=20Mhz; HT-protection=Legacy protect; Deny client=none;
AC=be; CWmin=4; CWmax=6; AIFS=3; Txoplimit=0; NoACK=disabled;
AC=bg; CWmin=4; CWmax=10; AIFS=7; Txoplimit=0; NoACK=disabled;
AC=vi; CWmin=3; CWmax=4; AIFS=1; Txoplimit=3008; NoACK=disabled;
AC=vo; CWmin=2; CWmax=3; AIFS=1; Txoplimit=1504; NoACK=disabled;
Rx packets=296832799; errors=  49058; dropped=    8;
Tx packets= 35562590; errors=4567503; dropped=30648;
Rx bytes=1675601068 (1.561 GB); Tx bytes=2871104053 (2.674 GB);
ACSP use last selection=disabled;
BGSCAN allow=enabled; BGSCAN during voice=disabled; BGSCAN interval=20 minutes;
BGSCAN with client=enabled; BGSCAN with PS client=disabled;
Number of BGSCAN=30911; Number of BGSCAN requested=36029; Number of BGSCAN misse                                                                                                                                                             d=5118;
DFS=disabled; Number of detected radar signals=0; DFS static-channel restore=dis                                                                                                                                                             abled;
LLDP state=N/A; CDP state=N/A;
Rx airtime=3234.11 s; Tx airtime=17740.56 s; CRC error airtime=1896.51 s;
Rx airtime percent=0.01%; Tx airtime percent=0.05%; CRC error airtime percent=0.                                                                                                                                                             29%;
Tx utilization=1%; Rx utilization=3%; Interference utilization=0%; Total utiliza                                                                                                                                                             tion=4%;
Backhaul failover=disable;
Radio-share mode=off; channel-change=disable;
Running average Tx CU=2%; Rx CU=4%; Interference CU=2%; Noise=-95dBm;
Short term means average Tx CU=1%; Rx CU=4%; Interference CU=1%; Noise=-95dBm;
Snapshot Tx CU=1%; Rx CU=4%; Interference CU=2%; Noise=-95dBm;
CRC error rate=49%;
Benchmark 11a score=222000; 11b score=50500; 11g score=222000; 11n score=501000;                                                                                                                                                              11ac score=889000; 11ax score=889000;
OFDMA downlink=disabled;
OFDMA uplink=disabled;
TWT=disabled;
BSS Color=0;
A-MSDU=disabled; A-MPDU limit=1048575;
VHT support in 2.4G interface=disabled;
Frameburst=enabled;
Alternate radio profile=disabled;
Primary radio profile=default_2.4ghz_gn;
DSSS 20Mhz power:
Board limit=20dBm(1) 20dBm(2) -(3) -(4); Regulatory limit=18dBm(1) 19dBm(2) -(3)                                                                                                                                                              -(4);
Spectral scan=off;
The signal by pass BPF.
 

I am currently noticing a very high crc rate. If i check the acsp neighbors he can see the other aps in the street maybe too good?
 

Bssid           Mode     Ssid/Hive                Chan Rssi(dBm) Aerohive AP  CU  CRC STA Channel-width VID  NVID
bcf3:103f:2dd4  Access   8021X                    1    -81       yes          0   0   0   20            6    1
bcf3:103f:2dd5  Access   MDM                      1    -81       yes          0   0   0   20            6    1
bcf3:103f:2dd6  Access   PPSK                     1    -81       yes          0   0   0   20            6    1
bcf3:103f:2c96  Access   PPSK                     6    -84       yes          0   0   0   20            6    1
bcf3:103f:2c94  Access   8021X                    6    -84       yes          0   0   0   20            6    1
bcf3:103f:2c95  Access   MDM                      6    -84       yes          0   0   0   20            6    1
bcf3:103f:1814  Access   8021X                    6    -61       yes          0   0   0   20            6    1
bcf3:103f:1815  Access   MDM                      6    -60       yes          0   0   0   20            6    1
bcf3:103f:1816  Access   PPSK                     6    -59       yes          0   0   1   20            6    1
bcf3:103f:1f54  Access   PPSK                     1    -60       yes          0   0   0   20            6    1
bcf3:103f:1f55  Access   8021X                    1    -58       yes          0   0   0   20            6    1
bcf3:103f:1f56  Access   MDM                      1    -62       yes          0   0   0   20            6    1
bcf3:103f:0c16  Access   PPSK                     6    -80       yes          0   0   0   20            6    1
bcf3:103f:0c14  Access   8021X                    6    -81       yes          0   0   0   20            6    1
bcf3:103f:0c15  Access   MDM                      6    -80       yes          0   0   0   20            6    1
bcf3:103f:6394  Access   8021X                    11   -80       yes          0   0   0   20            6    1
bcf3:103f:6395  Access   MDM                      11   -80       yes          0   0   0   20            6    1
bcf3:103f:6396  Access   PPSK                     11   -80       yes          0   0   1   20            6    1
bcf3:103f:2bd5  Access   MDM                      11   -84       yes          0   0   0   20            6    1
bcf3:103f:1994  Access   8021X                    11   -83       yes          0   0   0   20            6    1
bcf3:103f:1995  Access   MDM                      11   -83       yes          0   0   0   20            6    1
bcf3:103f:1996  Access   PPSK                     11   -83       yes          0   0   0   20            6    1
bcf3:103f:2756  Access   PPSK                     6    -89       yes          0   0   1   20            6    1
bcf3:103f:2295  Access   MDM                      11   -91       yes          0   0   0   20            6    1
bcf3:1062:36d6  Access   MDM                      11   -87       yes          0   0   0   20            6    1
bcf3:103f:2bd4  Access   8021X                    11   -85       yes          0   0   0   20            6    1
bcf3:103f:2bd6  Access   PPSK                     11   -84       yes          0   0   1   20            6    1
bcf3:1062:36d4  Access   8021X                    11   -87       yes          0   0   0   20            6    1
bcf3:103f:2296  Access   PPSK                     11   -90       yes          0   0   0   20            6    1
bcf3:103f:2294  Access   8021X                    11   -90       yes          0   0   0   20            6    1
bcf3:103f:1894  Access   8021X                    1    -86       yes          0   0   0   20            6    1
bcf3:103f:1896  Access   PPSK                     1    -86       yes          0   0   0   20            6    1
bcf3:103f:1b95  Access   MDM                      1    -84       yes          0   0   0   20            6    1
bcf3:103f:1b94  Access   8021X                    1    -83       yes          0   0   0   20            6    1
bcf3:103f:1b96  Access   PPSK                     1    -89       yes          0   0   0   20            6    1
bcf3:1062:36d5  Access   PPSK                     11   -87       yes          0   0   0   20            6    1
bcf3:103f:1895  Access   MDM                      1    -86       yes          0   0   0   20            6    1
bcf3:103f:1094  Access   8021X                    11   -86       yes          0   0   0   20            6    1
bcf3:103f:1095  Access   MDM                      11   -86       yes          0   0   0   20            6    1
bcf3:103f:1096  Access   PPSK                     11   -87       yes          0   0   0   20            6    1
bcf3:103f:1997  Access   Stapler                  11   -79       yes          0   0   0   20            6    1
bcf3:103f:1b97  Access   Stapler                  1    -87       yes          0   0   0   20            6    1
bcf3:103f:0c17  Access   Stapler                  6    -82       yes          0   0   0   20            6    1
bcf3:103f:2bd7  Access   Stapler                  11   -81       yes          0   0   0   20            6    1
bcf3:103f:1817  Access   Stapler                  6    -60       yes          0   0   0   20            6    1
bcf3:1062:36d7  Access   Stapler                  11   -89       yes          0   0   0   20            6    1
bcf3:103f:2797  Access   Stapler                  11   -90       yes          0   0   0   20            6    1
bcf3:103f:2c97  Access   Stapler                  6    -84       yes          0   0   0   20            6    1
bcf3:103f:2297  Access   Stapler                  11   -88       yes          0   0   0   20            6    1
bcf3:103f:2757  Access   Stapler                  6    -85       yes          0   0   0   20            6    1
bcf3:103f:1f57  Access   Stapler                  1    -59       yes          0   0   0   20            6    1
bcf3:103f:1097  Access   Stapler                  11   -86       yes          0   0   0   20            6    1
bcf3:103f:6397  Access   Stapler                  11   -81       yes          0   0   0   20            6    1
bcf3:103f:2dd7  Access   Stapler                  1    -81       yes          0   0   0   20            6    1
bcf3:103f:1897  Access   Stapler                  1    -85       yes          0   0   1   20            6    1
bcf3:103f:2755  Access   MDM                      6    -87       yes          0   0   0   20            6    1
bcf3:1068:f797  Access   Stapler                  6    -86       yes          0   0   0   20            6    1
bcf3:103f:1157  Access   Stapler                  11   -88       yes          0   0   0   20            6    1
bcf3:103f:1156  Access   PPSK                     11   -89       yes          0   0   0   20            6    1
bcf3:1058:7797  Access   Stapler                  1    -88       yes          0   0   0   20            6    1
bcf3:103f:1155  Access   MDM                      11   -90       yes          0   0   0   20            6    1
bcf3:103f:1154  Access   8021X                    11   -89       yes          0   0   0   20            6    1
bcf3:1058:7796  Access   MDM                      1    -90       yes          0   0   0   20            6    1
bcf3:103f:2754  Access   8021X                    6    -87       yes          0   0   0   20            6    1
bcf3:103f:2794  Access   8021X                    11   -90       yes          0   0   0   20            6    1
bcf3:1068:f796  Access   PPSK                     6    -88       yes          0   0   0   20            6    1
bcf3:1058:7794  Access   PPSK                     1    -88       yes          0   0   0   20            6    1
bcf3:1058:7795  Access   8021X                    1    -88       yes          0   0   0   20            6    1
bcf3:103f:2795  Access   MDM                      11   -90       yes          0   0   0   20            6    1
bcf3:103f:2796  Access   PPSK                     11   -91       yes          0   0   0   20            6    1
bcf3:1068:f795  Access   MDM                      6    -87       yes          0   0   0   20            6    1
bcf3:1068:f794  Access   8021X                    6    -87       yes          0   0   0   20            6    1
bcf3:1068:f8d7  Access   Stapler                  1    -90       yes          0   0   0   20            6    1
bcf3:1068:f8d5  Access   MDM                      1    -91       yes          0   0   0   20            6    1
10e7:c657:7063  Access   DIRECT-62-HP OfficeJe... 6    -70       no           --  --  --  20            -    -
8807:4bb2:1f80  Access   LG V10                   6    -59       no           --  --  --  20            -    -
bcf3:1068:f8d4  Access   8021X                    1    -89       yes          0   0   0   20            6    1
bcf3:1068:f8d6  Access   PPSK                     1    -93       yes          0   0   0   20            6    1
bcf3:1062:6797  Access   Stapler                  6    -87       yes          0   0   0   20            6    1
 

I guess the power rate on all Aps is still too high? I noticed many of the aps got a pretty high crc rate, even though their summary state says “fair”. Shouldn’t they say “high collision” like written in this guide? https://extremeportal.force.com/ExtrArticleDetail?an=000079422

F

Well  ok. Seems like the community page just deletes replies when they are above 3000 letters so my reply just got deleted before it was posted. So i will keep this one a little shorter. 

The motorolas used external antennas. 

 

Four

 

So i checked the acsp neighbor stats from my aps. Most of them can see one / two aps within the range of -50 to -65, the othe aps had above -70 / -80 dbm.

 

But i came across some aps like this one (he was in channel 11 on 2,4)

Bssid           Mode     Ssid/Hive                Chan Rssi(dBm) Aerohive AP  CU                                                                                                                                                               CRC STA Channel-width VID  NVID
bcf3:103f:3f14  Access   8021X                    11   -86       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:3f15  Access   MDM                      11   -86       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:3f16  Access   PPSK                     11   -86       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:2794  Access   8021X                    11   -68       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:2795  Access   MDM                      11   -68       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:2796  Access   PPSK                     11   -68       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:1b94  Access   8021X                    1    -83       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:1b95  Access   MDM                      1    -86       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:1b96  Access   PPSK                     1    -87       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:2dd4  Access   8021X                    1    -80       yes          0                                                                                                                                                                0   0   20            6    1
bcf3:103f:2dd5  Access   MDM                      1    -80       yes          0   0   0   20            6    1
bcf3:103f:2dd6  Access   PPSK                     1    -80       yes          0   0   0   20            6    1
bcf3:103f:2294  Access   8021X                    11   -65       yes          0   0   0   20            6    1
bcf3:103f:2296  Access   PPSK                     11   -65       yes          0   0   0   20            6    1
bcf3:103f:2295  Access   MDM                      11   -65       yes          0   0   0   20            6    1
bcf3:103f:32d4  Access   8021X                    6    -77       yes          0   0   0   20            6    1
bcf3:103f:32d5  Access   MDM                      6    -78       yes          0   0   0   20            6    1
bcf3:103f:32d6  Access   PPSK                     6    -79       yes          0   0   0   20            6    1
bcf3:103f:6394  Access   8021X                    11   -77       yes          0   0   0   20            6    1
bcf3:103f:6395  Access   MDM                      11   -77       yes          0   0   0   20            6    1
bcf3:103f:6396  Access   PPSK                     11   -77       yes          0   0   0   20            6    1
bcf3:1062:6795  Access   PPSK                     6    -82       yes          0   0   0   20            6    1
bcf3:1062:6796  Access   MDM                      6    -82       yes          0   0   0   20            6    1
bcf3:1062:6794  Access   8021X                    6    -82       yes          0   0   0   20            6    1
bcf3:103f:6695  Access   MDM                      11   -89       yes          0   0   0   20            6    1
bcf3:103f:6696  Access   PPSK                     11   -88       yes          0   0   1   20            6    1
bcf3:103f:6694  Access   8021X                    11   -88       yes          0   0   0   20            6    1
bcf3:103f:2c95  Access   MDM                      6    -83       yes          0   0   0   20            6    1
bcf3:103f:2c94  Access   8021X                    6    -83       yes          0   0   0   20            6    1
bcf3:103f:2c96  Access   PPSK                     6    -83       yes          0   0   0   20            6    1
bcf3:103f:1b97  Access   Stapler                  1    -86       yes          0   0   0   20            6    1
bcf3:103f:2797  Access   Stapler                  11   -71       yes          0   0   0   20            6    1
bcf3:1062:6797  Access   Stapler                  6    -82       yes          0   0   0   20            6    1
bcf3:103f:2c97  Access   Stapler                  6    -83       yes          0   0   0   20            6    1
bcf3:103f:2297  Access   Stapler                  11   -72       yes          0   0   0   20            6    1
bcf3:103f:6397  Access   Stapler                  11   -74       yes          0   0   0   20            6    1
bcf3:103f:3f17  Access   Stapler                  11   -85       yes          0   0   0   20            6    1
bcf3:103f:2dd7  Access   Stapler                  1    -79       yes          0   0   0   20            6    1
bcf3:103f:6697  Access   Stapler                  11   -88       yes          0   0   0   20            6    1
bcf3:103f:32d7  Access   Stapler                  6    -76       yes          0   0   0   20            6    1
bcf3:103f:4195  Access   MDM                      1    -90       yes          0   0   0   20            6    1
bcf3:103f:1897  Access   Stapler                  1    -86       yes          0   0   0   20            6    1
bcf3:103f:1896  Access   PPSK                     1    -86       yes          0   0   0   20            6    1
bcf3:1058:7797  Access   Stapler                  1    -88       yes          0   0   0   20            6    1
bcf3:1058:7795  Access   8021X                    1    -87       yes          0   0   0   20            6    1
bcf3:1058:7796  Access   MDM                      1    -89       yes          0   0   0   20            6    1
bcf3:1058:7794  Access   PPSK                     1    -87       yes          0   0   1   20            6    1
bcf3:1068:fb57  Access   Stapler                  11   -88       yes          0   0   0   20            6    1
bcf3:103f:1895  Access   MDM                      1    -86       yes          0   0   0   20            6    1
bcf3:103f:1894  Access   8021X                    1    -86       yes          0   0   0   20            6    1
bcf3:1068:fb56  Access   PPSK                     11   -89       yes          0   0   0   20            6    1
bcf3:1068:fb54  Access   8021X                    11   -88       yes          0   0   0   20            6    1
bcf3:1068:fb55  Access   MDM                      11   -89       yes          0   0   1   20            6    1
bcf3:103f:1817  Access   Stapler                  6    -88       yes          0   0   1   20            6    1
bcf3:103f:1816  Access   PPSK                     6    -89       yes          0   0   2   20            6    1
bcf3:103f:1814  Access   8021X                    6    -88       yes          0   0   1   20            6    1
bcf3:103f:1815  Access   MDM                      6    -87       yes          0   0   1   20            6    1
bcf3:103f:4197  Access   Stapler                  1    -88       yes          0   0   0   20            6    1
bcf3:103f:2e17  Access   Stapler                  11   -88       yes          0   0   0   20            6    1
bcf3:103f:2e14  Access   8021X                    11   -89       yes          0   0   0   20            6    1
bcf3:103f:2e16  Access   PPSK                     11   -91       yes          0   0   0   20            6    1
 

Why did he chose channel 11 even though the other two aps who are nearby are on the same. Is it because they have a too low signal strength for him to change it and he doesn’t see a problem in it?

 

I will try to get a packet capture and will visit the plant in two weeks to get some more information from the client side. 

O
Rank
Userlevel 5

Were you using external antennas on AP5131? How high are the APs mounted?

How many SSIDs do you have in total?

From the RF plan perspective, it does not look that bad apart from the upper right corner of the screenshot where most of the APs are using channel 6 (purple area). But again, this is not 100% realistic and we need to have some RF packet capture taken from a client to figure out if there are excessive retransmissions of the frames probably due to the height of the APs, integrated antenna, and metallic objects in the warehouse.

 

Some helpful commands to verify channel/power issues:

#sho int // to see BSSID

#sho hive // to see hive name which can be used to identify Cloud APs on the same channel

#sho acsp neigh // to see background scan list and then look for VHM hive name to easily identify BSSID of Cloud AP on the same channel

This should give you an output similar to the following, that will allow you to see how many of your APs are on the same channel and how well can they see each other, if its too strong of a signal you have too many APs (or high Tx power) and you should either reduce the Tx power or turn off some of the APs (but I doubt this would be the case), if the signal is too low you have less APs. 

 

wifi1(6) ACSP neighbor list (17/384): Bssid           Mode     Ssid/Hive                Chan Rssi(dBm) Aerohive AP  CU  CRC STA Channel-width VID  NVID be4d:fbbf:9860  Access   3sCompany-5G             36   -87       no           --  --  --  40+           -    -    206c:8a05:1924  Access   AOA                      36   -58       yes          7   17  2   20            1    1    206c:8a05:1920  Backhaul VHM-CYRUJRGN             36   -57       yes          7   17  1   20            1    1    989d:5de9:77aa  Access   kikobull-5G              40   -77       no           --  --  --  80            -    -    2405:8823:8f8b  Access   google                   149  -85       no           --  --  --  80            -    -    2405:8823:8546  Access   google                   149  -62       no           --  --  --  80            -    -    2405:8823:9b67  Access   google                   149  -75       no           --  --  --  80            -    -    3894:edf2:c882  Access   NETGEAR77-5G             149  -77       no           --  --  --  80            -    -    4065:a3ab:1097  Access   BELL918                  157  -89       no           --  --  --  80            -    -    989d:5de9:77ab  Access                            40   -76       no           --  --  --  80            -    -    989d:5de9:77af  Access                            40   -76       no           --  --  --  80            -    -    989d:5de9:77ad  Access                            40   -76       no           --  --  --  80            -    -    989d:5de9:77b0  Access                            40   -76       no           --  --  --  80            -    -    2605:8823:8f89  Access                            149  -86       no           --  --  --  80            -    -    2605:8823:8545  Access                            149  -61       no           --  --  --  80            -    -    bcf3:102b:80b0  Backhaul hive0                    36   -48       yes          0   0   0   20            1    1    36f1:50c4:47c0  Access   DIRECT-roku-L0J-417A8... 36   -54       no           --  --  --  20            -    -

To solve this issue, you need to first determine if it the application disconnect or client disassociation/disconnect. if its client disassociation/disconnect, the cause could be roaming, bad coverage etc. If its only application disconnect, it could be due to bad RF, retransmission to being with, and the best way to check it would be to connect a client to an AP, don't move the client and check if the application still disconnects. 

While doing it take packet capture, without it we can’t determine the issues in an effective way. 

Regards,

Ovais    

D
Userlevel 2
Badge +1

You could try to reduce the radio power to 13-15 db and check it again

Reply