Author Topic: Is Speedify a SDN? How does it exactly work?  (Read 690 times)

readme (OP)

  • Bit
  • *
  • Join Date: May 2021
  • Posts: 1
  • Rep: 0
    • View Profile
Is Speedify a SDN? How does it exactly work?
« on: May 08, 2021, 10:09:09 AM »
Hi, great forum, my first post!

I'm trying to figure out an alternative, Fortinet solutions seems to be very limited, and Peplink's behavior isn't suitable for mixed usages. Please correct me if I'm wrong. I've read a topic on this forum regarding "Riverbed" which sparked my interest.

I've collected some results in a brief way to reproduce the behavior (before the questions) based on samples over few days, for those who haven't tested it:
Setup:
Latency to ServerBWTypeNAT?Bufferbloat on loadNotes
ISP172ms avg14/1 MbitADSLDouble+100msLow UDP throughput
ISP281ms avg12/12 MbitWireless/PIADedicated Pub IP + Open ports+12msTCP is out of order


Test Scenario:
Latency AvgJitterFile Download SpdTotal Throughput Avg
Segmented file download test + Video call182ms21ms23.7 Mbit24.5 Mbit
Same as above with "packetaggr = off"178ms20ms23.8 Mbit24.5 Mbit
Single TCP file download test + Video call90ms (HTTPING/TCP=300ms)12ms21.2 Mbit22.3 Mbit
Same as above with "packetaggr = off"78ms14ms13 Mbit (ADSL was only active, VoIP on ISP2 was active)14 Mbit


There is a 5 second delay before the single TCP file download test is boosted, it starts at 13 Mbit pulling ~6 Mbit from each connection then it ramps up after 5 seconds totaling ~21 Mbit,(~11 Mbit from each). It's a bit spikey though, for every second 19-24, while segmented is smooth 23.

Looking at Speedify's installation directory there seems to be nDPI libraries included. It's probably used to detected streaming connections, as it had no problem detecting video games, and any constant low bitrate UDP connection without DSCP markings, or L7. Speedify shows the stream's name and it seems to match the app list from Ntopng, then route it to the redundant channel.

Looking at the results as in speculation, it seems like there are three channels being used, a packet aggregation channel, load balancing channel/proxy, and a redundant channel running all simultaneously; It is able to route connections with the help of nDPI to any of these channels. What I can confirm regarding the segmented download result is that Speedify tries to avoid packet aggregation as much as possible, especially if the connections are the same/duplicated as in segmented download, I was able to confirm this by simply detaching one of the ISPs, the segmented download conns. numbers is 8, 3 went down for split second and up again. The delay in the single TCP speed test ramp up confirms it, so maybe there is some form of steering that is involved (Load balance being useless for single unless total is saturated -> packet aggregation channel).
HTTPING vs ICMP/UDP ping in packet aggregation mode has a large difference, seems like only TCP is buffered/reordered, while the rest aren't.

There is also the possibility of packet aggregating a stream if they exceed a certain speed of the slowest connection such as Twitch in my test, ISP1 only having 1 Mbit while Twitch 3-5 Mbit, the UI will show "Streaming with bonding" instead of "Streaming with redundant" to boost single upload, but other low bit rate streams will stick to redundant. Having two ISPs at 4+ Mbit upload will stick to redundant for Twitch as I tested with LTE, makes sense if one of the 2 ISPs that is up only providing 1 Mbit, well below Twitch's common bitrate of 3-5 Mbit.

It also has the ability to use different protocol per uplink(ISP): UDP, TCP, Multi TCP, and HTTPS (for disguise).
For multi TCP, the max limit is 8 connections per interface, selecting this protocol manually forces the max limit, while Auto mode defaults to Multi TCP on latency > 60ms, else UDP is used. Auto also adjusts the Multi TCP connections depending on buffer bloat? It lowered the number to 2 connections on ISP1 (ADSL), 8 connections on ISP2. Forcing 8 on ADSL lowered the interface speed by around 2 Mbit. Comparing Single TCP protocol to Multi TCP in this setup showed no marginal difference other than the upload speed on ISP2 is fully saturated. (11Mbit Multi vs 9 Mbit Single)
Another aspect with Speedify used for a single connection with multi TCP is long distance connections, I haven't tested BBR yet (their servers do have bbr enabled), Netherlands to California in TCP mode (like any VPN) with ISP2 averaged 6 Mbit; 12 Mbit with Multi TCP.

Encryption enabled (default) with a very lossy link (simulated with a WiFi AP) improved throughput at the cost of latency in comparison. It also has the ability to specify encryption per interface from CLI. This option seems to be included for ISPs that filters/blocks VPNs. My ISP1 performs very poorly with any chacha20 UDP VPN incl. Wireguard, AES is unaffected, and disabling encryption for ISP1 improves the result for UDP mode at full speed. Speedify prefers AES whenever available. UDP mode was poor on ISP2 regardless of any encryption, seems like TCP is hardware accelerated from the middle boxes of ISP2.

On upload, Speedify completely ignored ADSL for balancing or/and aggregation upstream, lowering the "overflowthreshold" value forces it, but the difference between the two ISPs is too large, was only pulling 0.5 Mbit from ADSL and increased latency, not worth it (1 vs 12!). Adding traffic shaped 3 Mbit upload LTE worked fine with no changes other than the increased speed.

Removing or adding a new connection or simulating packet loss did not cause any micro stuttering to video calls or games at 300 ticks (10ms interval ICMP), and the single TCP file download speed simply lowered with a 100ms extra latency spike for a split second (measured using httping for TCP), in comparison with packetaggr mode off however there is a ~500ms packet loss/DC period before it switches to the second connection. SSH and other low latency TCP services (websocket) didn't show any changes with one of the ISP's being ripped off, however with packetaggr off it lagged for a split second. Packet aggregation is enabled by default, and the flag is hidden in CLI.

Tl;dr: This was all ~kinda poorly~ done testing to speculate the inner workings, the question is the definition for this implementation, is it WAN smoothing? lightweight SD-WAN? or as Speedify calls it channel-bonding, if so any channel-bonding spec/implementaion with DPI I've missed? I'm pretty new to SD-WAN and SDNs, and I'm looking for a self-hosted alternative.

Thanks for reading!

« Last Edit: May 08, 2021, 10:43:11 AM by readme »

deanwebb

  • Permit any any all log
  • Administrator
  • OC-1920
  • *****
  • Join Date: Jan 2015
  • Posts: 9862
  • Country: us
  • Rep: 29
  • *I* am the one who NACs.
    • View Profile
  • Certifications: FCSE #273: ForeScout Certified System Engineer, Tufin CSE, TippingPoint ASE, MOSMWNMTK
Re: Is Speedify a SDN? How does it exactly work?
« Reply #1 on: May 10, 2021, 10:52:33 AM »
Lots of data... what's the use case here?

Sorry, I'm a security guy, so stuff outside my core is not 100% obvious to me.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!

Otanx

  • Senior Engineer
  • ****
  • Join Date: Jan 2015
  • Posts: 2293
  • Country: us
  • Rep: 21
    • View Profile
  • Certifications: CCNP
Re: Is Speedify a SDN? How does it exactly work?
« Reply #2 on: May 10, 2021, 11:44:54 AM »
the question is the definition for this implementation, is it WAN smoothing? lightweight SD-WAN? or as Speedify calls it channel-bonding, if so any channel-bonding spec/implementaion with DPI I've missed? I'm pretty new to SD-WAN and SDNs, and I'm looking for a self-hosted alternative.

So yes it can do WAN smoothing, it can be considered SD-WAN (I do). I don't know what they are doing under the hood. Probably the best way to figure it out would be packet captures of their outside interfaces. If I had to guess they are building tunnels over each outside interface back to their cloud. Then they can do their cool guy stuff using those tunnels. Want to do redundancy? They send every packet down both paths, and dedup on their system in the cloud. Want to aggregate bandwidth? They just load balance over the tunnels. They could even do some kind of weighted per-packet load balance so even a single flow would get better.

I am not sure what you mean by a self-hosted alternative. What requirements do you have? Speedify does quite a bit, and I don't think you will find a self-hosted solution off the shelf.

-Otanx