Networking-Forums.com

General Category => Forum Lobby => Topic started by: Dieselboy on August 02, 2016, 12:43:26 AM

Title: unlucky :( mem leak
Post by: Dieselboy on August 02, 2016, 12:43:26 AM
I am aware of a bug in my current IOS code to do with SSL VPN and a memory leak.
Last weekend, I planned a reload to clear a leak (Saturday). Up time was 4 months as we had a complete power down 4 months ago to install server room AC.

Got an alert this morning (Tuesday) that free mem is below my configured threshold. I've been watching free mem drop off all day :(

Don't think I'll make it to the weekend.
Title: Re: unlucky :( mem leak
Post by: deanwebb on August 02, 2016, 09:02:32 AM
I don't think you'll make it to Wednesday.
Title: Re: unlucky :( mem leak
Post by: Dieselboy on August 02, 2016, 09:15:48 AM
After posting this I confirmed that the memory leak was not due to a previous bug as this IOS has resolved that bug - so this is a new memory leak to me.

I done an upgrade to 1 release up, went from T3 to T4 release with fingers crossed. I cannot go any newer because Cisco have implemented a fix / enhancement into SSL VPN but failed to think about SSL VPN phones. Consequently this fix breaks SSL VPN phones. Going to speak to my account manager tomorrow for options.
Title: Re: unlucky :( mem leak
Post by: Dieselboy on August 02, 2016, 11:35:36 PM
I sent a lengthy email, politely worded to our Cisco AM, and given 8 TAC cases I've raised over the past few years which resulted in the memory leaks etc, 6 CSC bugs filed and 1 Field Notice. The other case is open still as raised yesterday.

I have asked for one of three options:
1. An IP Phone firmware engineering special, that I can use for our remote VPN phones to support the "fix" in the IOS SSL VPN. I Think this is a reasonable request, I'm not aware of the scope required in Cisco's back end though.

2. An IOS engineering special that does not include the "fix" but allows me to upgrade the router to mitigate known vulnerabilities. -> I don't want to do this for a number of reasons, but it's an option.

3. A free ASA-X AnyConnect IP phone license so I can move the phones to the ASAs. This is my option 2. if option 1 is not achievable.

May move this thread to another section if it gets more techy.
Title: Re: unlucky :( mem leak
Post by: Dieselboy on August 04, 2016, 04:35:12 AM
It's happening again
Title: Re: unlucky :( mem leak
Post by: deanwebb on August 04, 2016, 11:50:37 AM
Love the graphic title... and I'm thinking that migrating to the ASA may be the solution the Cisco guys might favor.
Title: Re: unlucky :( mem leak
Post by: icecream-guy on August 04, 2016, 01:41:32 PM
Quote from: deanwebb on August 04, 2016, 11:50:37 AM
Love the graphic title... and I'm thinking that migrating to the ASA may be the solution the Cisco guys might favor.

ASA code ain't too clean either. at least the 9.1,  not sure about the higher rev trains.
Title: Re: unlucky :( mem leak
Post by: Dieselboy on August 04, 2016, 09:11:35 PM
I was monitoring this last night whilst clenching the buttocks. RAM use has levelled out with 85mb free. Will keep an eye on it, but panick over for the moment.

Graph title was extreme irony ;) I sent this to the tac engineer, hope she found it funny too. I did explain in the email that I made a very rough prediction though, in case my humour was lost in context / translation.
:awesome:

>:D
Title: Re: unlucky :( mem leak
Post by: wintermute000 on August 05, 2016, 01:23:45 AM
It could be worse. I once had a 3750X (IOS 15, natch) whose RAM leak locked out the AAA process.... you could see a memory alloc error on the AAA process on the console if you tried to (and failed) SSH. Had to physically reboot, couldn't even get in via console.
Title: Re: unlucky :( mem leak
Post by: Dieselboy on August 05, 2016, 03:24:37 AM
That's bad. I had that happen to this device but I don't use AAA for SSH access. I use RADIUS for the SSL VPN authentication and I had the exact same issue - no one could log in to the VPN on this box with the same error log.

So in short - don't add complications to your management process ;)