Why is freebsd-update so horrible slow?

[Erik Lauritsen]

I have been setting here for about 6 hours waiting for an upgrade from 12.2 to 13 and freebsd-update is only half way through getting all the patches!

It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.

Then I did a search and this has apparently been an issue for several years, it has even been filed as a bug.

Anyway, I have cancelled the upgrade. If it's going to take this long, I am not going to upgrade this, but will most likely replace it with another OS.

[X Guest]

maybe you have slow network connection to the update servers.
you can give a try on checking the connection such as ping and traceroute.

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade from 12.2 to 13 and freebsd-update is only half way through getting all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this long, I am not going to upgrade this, but will most likely replace it with another OS.
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions

--
https://xguest.net/

[Isak Holmström via freebsd-questions]

Post by X Guest
maybe you have slow network connection to the update servers.
you can give a try on checking the connection such as ping and traceroute.

It might be overloaded servers as well. Maybe try in a day or two. But 6 hours, that’s long. For me only about an hour.

/Isak

Post by X Guest

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade from 12.2 to 13 and freebsd-update is only half way through getting all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this long, I am not going to upgrade this, but will most likely replace it with another OS.
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions

--
https://xguest.net/
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions

[Jacques Foucry]

Post by Isak Holmström via freebsd-questions

Post by X Guest
maybe you have slow network connection to the update servers.
you can give a try on checking the connection such as ping and traceroute.

It might be overloaded servers as well. Maybe try in a day or two. But 6 hours, that’s long. For me only about an hour.

About 1 hour for me too.

[Doug Hardie]

Post by Jacques Foucry

Post by Isak Holmström via freebsd-questions

Post by X Guest
maybe you have slow network connection to the update servers.
you can give a try on checking the connection such as ping and traceroute.

It might be overloaded servers as well. Maybe try in a day or two. But 6 hours, that’s long. For me only about an hour.

About 1 hour for me too.
--

I updated 5 machines yesterday after receiving the announcement. Three of them took about 5 minutes. One took about an hour, and the last about 3 hours. I didn't keep track of which server was used, but I believe the slowest was on server 4. Also, the time required was dependent on the need to update the source. The number of update files was more than about 5 times if source had to be updated. The fastest updates were for the machines that were at 13.0-RC5. The slower were for those on 12.x except for the the last one which did not update source.

-- Doug

[Gökşin Akdeniz]

Hello,

I suppose my humble opinions and advices may help.

Download speeds depends vary widelydepending of many actors. LAN speed
do not guarantee the "full-speed". I have gigabit LAN but ISP offeres
IPv4 connection with 100 Mbps which I can experience 80 Mbps at ideal
conditions. "Usual speed" for FreeBSD-UPDATE servers onrelease dates is
about at 50-60 Mbps. I wait for a couple of days before I upgrade any
systems.

Besides, servers handling upgrade requests and network infrastructure
they are connected, may be overloaded and may lead to "poor performance".

Metwork performance may vary due to number of clients and hosts, routing
of packages, jitter, resending of packages du to TTL and package loss,
congesteg nodes and routes and etc.

I advise, waiting for a couple of days or so before attempting to upgrade.

And ypu may replace existing installations with any OS of your choice.

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade from 12.2 to 13 and freebsd-update is only half way through getting all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this long, I am not going to upgrade this, but will most likely replace it with another OS.
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions

[Steve O'Hara-Smith]

On Wed, 14 Apr 2021 02:39:39 +0100

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade from
12.2 to 13 and freebsd-update is only half way through getting all the
patches!

Overloaded network segment, or overloaded servers. I'd bet on the
latter.

Post by Erik Lauritsen
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.

You should have tried downloading Linux 1.0 any time in the week
after the announcement I saw "1 byte(s) per second" and gave up - that
release pretty much DDOS'd Finland.

[Ralf Mardorf]

Post by Isak Holmström via freebsd-questions
overloaded servers

In my experiences with different operating systems, servers (mirrors)
might be overloaded or the ISP arbitrarily restricts download
speed or something ISP or router related is broken. Sometimes
restarting the router or if more hardware should be involved, such as
a separated DSL-splitter, disconnecting and reconnecting all
connections solves such issues. Usually just waiting a few minutes,
hours or days is all that is needed.

While I'm in favour of Linux over FreeBSD for my domains, I strongly
discourage to migrate away from FreeBSD to another BSD or Linux, just
because Internet download speed is fishy. I experiences this for
Windows running in a VM and my iPads, from time to time, too.

At least in Germany it's not unusual, that from time to time download
speed is catastrophic. Take a look at websites where users report
issues with ISPs. They sometimes provide graphs with reported
disturbances by hours.

Sometimes the last mile is just fishy. Usually I'm joking, "it's
raining, so the last mile can't work. I should have known better"! I
don't know how well they maintain local loops in your country. Here in
the Ruhrgebiet they seem not to maintain those at all ;).

And again, servers (mirrors) could be more or less down, too, even
without DDoS attacks involved.

[Ralf Mardorf]

Post by Ralf Mardorf
Sometimes the last mile is just fishy.

"The last mile is typically the speed bottleneck in communication
networks" - https://en.wikipedia.org/wiki/Last_mile

I suspect that "nice" for "altered scheduling priority" is the wrong
word here, probably "guileful" fits better, than "nice" does.

If the ISP does notice that you might cause lots of traffic to update
FLOSS, they might decide to kick you out and instead provide more
bandwidth for Corona meetings via Microsoft Teams.

[tech-lists]

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade
from 12.2 to 13 and freebsd-update is only half way through
getting all the patches!
It's painful. It's like the 90th with 56k modems and I am on a
gigabit wire.

It could be a whole host of things. Regardless of the speed of your
connection, it could be slowed by any congestion between your
connection and the update servers. It could be caused by dns timing
out on your connection. Or distance. Some diagnostic like traceroute
and ping would help.

An upgrade from 12.2-p6 to 13.0 took around 20 mins for me, and I'm on
fibre. That's freebsd-update fetch, install, reboot, install, reboot.

% host update.freebsd.org
update.freebsd.org has address 163.237.247.16
update.freebsd.org has address 204.15.11.117

the 1st ip is 20 hops away from me, the 2nd one 16 hops. My machines are
in the UK.

[Arthur Chance]

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade
from 12.2 to 13 and freebsd-update is only half way through getting
all the patches!
It's painful. It's like the 90th with 56k modems and I am on a
gigabit wire.
Then I did a search and this has apparently been an issue for several
years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this
long, I am not going to upgrade this, but will most likely replace it
with another OS.

This tweet from Colin Percival talks about the problem.

https://twitter.com/cperciva/status/1382491128537096193

TL;DR: changes to git caused a massive overload, the number of mirrors
has been increased to deal with it.

[Arthur Chance]

Post by Arthur Chance

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade
from 12.2 to 13 and freebsd-update is only half way through getting
all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several
years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this
long, I am not going to upgrade this, but will most likely replace it
with another OS.

This tweet from Colin Percival talks about the problem.
https://twitter.com/cperciva/status/1382491128537096193
TL;DR: changes to git caused a massive overload, the number of mirrors
has been increased to deal with it.

To clarify, I mean changing portsnap to use git, rather than changes in
git itself.

[Doug McIntyre]

Post by Arthur Chance

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade
from 12.2 to 13 and freebsd-update is only half way through getting
all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several
years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this
long, I am not going to upgrade this, but will most likely replace it
with another OS.

This tweet from Colin Percival talks about the problem.
https://twitter.com/cperciva/status/1382491128537096193
TL;DR: changes to git caused a massive overload, the number of mirrors
has been increased to deal with it.

Although his status is on portupgrade, not freebsd-update.

Is there any way for me to build my own update server locally? I've
tried web caching the data, but I have to be able to get all the data first..
(What the handbook mentions IIRC)

Fetching 3299 files... ....10....20....30....40....50....60....70....80....90....100....110....120..
..130....140....150....160....170 gunzip: (stdin): unexpected end of file
ae47b8b528bdf1369c7779d0dd453364922c77f3993fd147486b952fdbe91634 has incorrect hash.

At various points along the fetch on each try.

[RW via freebsd-questions]

On Thu, 15 Apr 2021 12:11:28 -0500

Post by Doug McIntyre

Post by Arthur Chance
This tweet from Colin Percival talks about the problem.
https://twitter.com/cperciva/status/1382491128537096193
TL;DR: changes to git caused a massive overload, the number of
mirrors has been increased to deal with it.

Although his status is on portupgrade, not freebsd-update.

On portsnap.

The freebsd-update and portsnap clients are so similar internally I'd be
surprised if they don't share server-side infrastructure.

[Daniel Stevenson]

Post by Doug McIntyre

Post by Arthur Chance

Post by Erik Lauritsen
I have been setting here for about 6 hours waiting for an upgrade
from 12.2 to 13 and freebsd-update is only half way through getting
all the patches!
It's painful. It's like the 90th with 56k modems and I am on a gigabit wire.
Then I did a search and this has apparently been an issue for several
years, it has even been filed as a bug.
Anyway, I have cancelled the upgrade. If it's going to take this
long, I am not going to upgrade this, but will most likely replace it
with another OS.

This tweet from Colin Percival talks about the problem.
https://twitter.com/cperciva/status/1382491128537096193
TL;DR: changes to git caused a massive overload, the number of mirrors
has been increased to deal with it.

Although his status is on portupgrade, not freebsd-update.
Is there any way for me to build my own update server locally? I've
tried web caching the data, but I have to be able to get all the data first..
(What the handbook mentions IIRC)
Fetching 3299 files... ....10....20....30....40....50....60....70....80....90....100....110....120..
..130....140....150....160....170 gunzip: (stdin): unexpected end of file
ae47b8b528bdf1369c7779d0dd453364922c77f3993fd147486b952fdbe91634 has incorrect hash.
At various points along the fetch on each try.

There's an article about it on docs.freebsd.org:

https://docs.freebsd.org/en/articles/freebsd-update-server/

It's a few years old, but it might still be useful.

--
Daniel Stevenson