UFS2 limits

[n***@people.net.au]

Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a single
directory.
This number will keep rising and I assume UFS2 has a limit to the number of
sub-directories in a single directory - can anyone tell me what it is?

What about ZFS?

At some point I'll have to re-arrange things so that I have a deeper directory
structure, just wondering when I'll hit the limit so I can plan in advance :-)

Cheers,
Ian

[Jeremy Chadwick]

This post might be inappropriate. Click to display it.

[Ian]

Post by Jeremy Chadwick

Post by n***@people.net.au
Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a
single directory.
This number will keep rising and I assume UFS2 has a limit to the number
of sub-directories in a single directory - can anyone tell me what it is?

As far as I know, there is no such limit on the number of files/dirs
inside of a directory.
I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.
If this is something you've written or have control over or can work
with engineers in regards to, I recommend you change your directory
naming scheme to have separate subdirectories with the first 2 or 3
/some/place/00/00ilikezeros/*
/some/place/01/01binaryheaven/*
/some/place/aa/aardvarks/*
/some/place/ab/abuse/*
/some/place/ac/actuary/*
...
/some/place/xy/xylophones/*
You get the point.
Traversing this structure is much more efficient, and requires very
little code change on your part. Those who run nameservers that host
many zones, for example, use this structure to ensure the daemon doesn't
take 32498231 years to start up.

Post by n***@people.net.au
What about ZFS?
At some point I'll have to re-arrange things so that I have a deeper
directory structure, just wondering when I'll hit the limit so I can plan
in advance :-)

What baffles me is why you're looking at this problem from a " "how can
the filesystem solve this engineering mistake I made for me" standpoint,
rather than "how can I solve this engineering mistake I made so that it
doesn't impact the filesystem". Very strange. Sometimes looking at
things in a different light makes all the difference.
Hope this helps.
P.S. -- I hope this mail makes it to you, because your From line is
that!).

Thanks for that Jeremy. I didn't invent this structure, but I daresay I can
either modify it or get the original writer to do that. I never really gave
it a thought before now - it was the system I was given to work with and it's
worked fine so far, except when I try to list the contents of the directory -
that takes ages!
All the folders are 7 digit numbers and we are up to approx. 0010500 entries
(ie subdirs) so far.
I guess it will just be a matter of experimenting to find the optimum number
of sub-sub-directories per sub-directory :-/

Oh, and yes, that email address does work - I use it for mailing lists and
other stuff where I'm likely to get spammed - it's ironic really :-)

Cheers,

[Chris Pratt]

Post by Ian

Post by Jeremy Chadwick

Post by n***@people.net.au
Hi,
I have a FreeBSD server that has about 10,500 subdirectories
within a
single directory.
This number will keep rising and I assume UFS2 has a limit to the number
of sub-directories in a single directory - can anyone tell me what it is?

As far as I know, there is no such limit on the number of files/dirs
inside of a directory.

Thanks for that Jeremy. I didn't invent this structure, but I
daresay I can
either modify it or get the original writer to do that. I never really gave
it a thought before now - it was the system I was given to work with and it's
worked fine so far, except when I try to list the contents of the directory -
that takes ages!
All the folders are 7 digit numbers and we are up to approx.
0010500 entries
(ie subdirs) so far.
I guess it will just be a matter of experimenting to find the
optimum number
of sub-sub-directories per sub-directory :-/

On the issue of possible inode limitations, it may be of some
reassurance
(or alarm ;-)) to look at the output of df -i. This will tell you if
you are close
to any limit on inodes. Between that and your already well known
counts of
directories and rate of creation, you can gauge how much time you
have to
change your app.

[H.fazaeli]

This post might be inappropriate. Click to display it.

[Erik Trulsson]

Post by Jeremy Chadwick

Post by n***@people.net.au
Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a single
directory.
This number will keep rising and I assume UFS2 has a limit to the number of
sub-directories in a single directory - can anyone tell me what it is?

As far as I know, there is no such limit on the number of files/dirs
inside of a directory.

Actually there is. Each i-node on the disk contains a field telling how
many hard-links point to that inode. This field is a (signed) 16-bit value,
meaning the maximum number of hardlinks allowed is 32767.
Each subdirectory created contains a hardlink ('..') to its parent, thus
limiting the number of subdirectories to a single directory to less than 32767.

Note that this does not limit the number of files you can have in a single
directory, since normal files do not contain hardlinks to the parent
directory, but there are of course limits to the total number of files and
directories you can have on a single filesystem based on how many inodes
were created when the filesystem was first created.

[Jeremy Chadwick]

Post by Erik Trulsson

Post by Jeremy Chadwick

Post by n***@people.net.au
Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a single
directory.
This number will keep rising and I assume UFS2 has a limit to the number of
sub-directories in a single directory - can anyone tell me what it is?

As far as I know, there is no such limit on the number of files/dirs
inside of a directory.

Actually there is. Each i-node on the disk contains a field telling how
many hard-links point to that inode. This field is a (signed) 16-bit value,
meaning the maximum number of hardlinks allowed is 32767.
Each subdirectory created contains a hardlink ('..') to its parent, thus
limiting the number of subdirectories to a single directory to less than 32767.

I hadn't even thought of that (the inode possibility did cross my mind,
but I figured inode counts would a) apply to the filesystem as a whole,
and b) apply to both files and directories.

Wow, thanks for the tip! Learning something new every day... :-)

Post by Erik Trulsson
Note that this does not limit the number of files you can have in a single
directory, since normal files do not contain hardlinks to the parent
directory, but there are of course limits to the total number of files and
directories you can have on a single filesystem based on how many inodes
were created when the filesystem was first created.

[Polytropon]

Post by Erik Trulsson
Note that this does not limit the number of files you can have in a single
directory, since normal files do not contain hardlinks to the parent
directory, but there are of course limits to the total number of files and
directories you can have on a single filesystem based on how many inodes
were created when the filesystem was first created.

Maybe this sounds stupid, but... given that a file system
can hold n entries. What happens when a program tries to
create file number n + 1?

I do ask this in order to explore if this could have been
the reason for my massive data loss and UFS file system
corruption.

[Erik Trulsson]

Post by Polytropon

Post by Erik Trulsson
Note that this does not limit the number of files you can have in a single
directory, since normal files do not contain hardlinks to the parent
directory, but there are of course limits to the total number of files and
directories you can have on a single filesystem based on how many inodes
were created when the filesystem was first created.

Maybe this sounds stupid, but... given that a file system
can hold n entries. What happens when a program tries to
create file number n + 1?
I do ask this in order to explore if this could have been
the reason for my massive data loss and UFS file system
corruption.

I haven't tested what actually happens, but what should happen is that the
attempt to create file n+1 will simply fail with some appropriate error code
(see open(2) or mkdir(2) for details.) It is certainly not supposed to
cause any kind of files system corruption.

[Roland Smith]

Post by Polytropon

Post by Erik Trulsson
Note that this does not limit the number of files you can have in a single
directory, since normal files do not contain hardlinks to the parent
directory, but there are of course limits to the total number of files and
directories you can have on a single filesystem based on how many inodes
were created when the filesystem was first created.

Maybe this sounds stupid, but... given that a file system
can hold n entries. What happens when a program tries to
create file number n + 1?

The call to the open(2) system call will fail for one of the reasons
given in the manual page.

Post by Polytropon
I do ask this in order to explore if this could have been
the reason for my massive data loss and UFS file system
corruption.

My first point of inquiry in such a case would be if the hardware is
OK. If you're using (P|S)ATA|SCSI-3 devices, install smartmontools from
ports and test the disk with smartctl(8).

Roland

[Pieter Donche]

I installed FreeBSD 7.0 already on a system at my office (from CDs)
and set up internet connection while installing using the office LAN
(and its DHCP service).

I installed FreeBSD also on my laptop at home (with no network connection).
(with X windows; startx gives me twm ..)
I want to install KDE3 from /usr/ports/x11/kde3
Tried as described: # cd /usr:ports/x11:kde3; make install clean
I had hoped everything necessary would be found in the disk.
But not: It returns errors because it tried to fetch packages from ftp
sites ... an soon stops..

So I still need Internet connection?

At home, I have an Internet connection via a Broadband Modem and an
account at an Internet Service Provider.
This was set up in WindowsXP succesfully (with IP 10.0.0.something,
netmask 255.255.255.0, no gateway, 2 DNS server IP addresses provided
by the ISP). When making the connection to my ISP I have to enter my
username at the ISP provider (***@PROVIDERNAME) and my password)

But how do I set up that in FreeBSD ??

In the Post-installation section during Setup, at the question
'Would you like to configure any Ethernet or SLIP/PPP network devides?)
answering Yes shows the correct on-board netwerk interface (Broadcom
BCM570xx PCI Gigabit ethernet card)

I tried DHCP, but no success (which was to expect)
Then I get the screen to fill in Host, Domain, Gateway, Name server
IPv4 address, netmask ...

What do I have to enter (Name server, netmask I can guess)
for IP?, host?, domain?
and where will come in my username/password for my ISP connection

Thanks for hints.
Is there any document describing such a connection?

pieter

[Fbsd1]

Post by Pieter Donche
I installed FreeBSD 7.0 already on a system at my office (from CDs)
and set up internet connection while installing using the office LAN
(and its DHCP service).
I installed FreeBSD also on my laptop at home (with no network
connection). (with X windows; startx gives me twm ..)
I want to install KDE3 from /usr/ports/x11/kde3
Tried as described: # cd /usr:ports/x11:kde3; make install clean
I had hoped everything necessary would be found in the disk.
But not: It returns errors because it tried to fetch packages from ftp
sites ... an soon stops..
So I still need Internet connection?
At home, I have an Internet connection via a Broadband Modem and an
account at an Internet Service Provider. This was set up in WindowsXP
succesfully (with IP 10.0.0.something,
netmask 255.255.255.0, no gateway, 2 DNS server IP addresses provided
by the ISP). When making the connection to my ISP I have to enter my
But how do I set up that in FreeBSD ??
In the Post-installation section during Setup, at the question
'Would you like to configure any Ethernet or SLIP/PPP network devides?)
answering Yes shows the correct on-board netwerk interface (Broadcom
BCM570xx PCI Gigabit ethernet card)
I tried DHCP, but no success (which was to expect)
Then I get the screen to fill in Host, Domain, Gateway, Name server
IPv4 address, netmask ...
What do I have to enter (Name server, netmask I can guess)
for IP?, host?, domain?
and where will come in my username/password for my ISP connection
Thanks for hints.
Is there any document describing such a connection?
pieter
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to

There is step by step instructions in the installer guide at
www.a1poweruser.com

[Matthew Seaman]

Post by Jeremy Chadwick
I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.

With the implementation of UFS_DIRHASH the practical limit on the
size of directories is now a great deal larger. In particular
the slow down caused by linear search through the contents has been
eliminated. See ffs(7). 10,000 files or sub-directories, whist
not a particularly elegant setup, is actually not unworkable
nowadays.

As for the maximum number of subdirectories it is possible to create
on UFS2 -- it is limited by the inode structure to a 16 bit quantity.

% jot 100000 1 | xargs mkdir -v
[...]
32725
32726
32727
32728
32729
32730
3273mkdir: 32766: Too many links
mkdir: 32767: Too many links
mkdir: 32768: Too many links
mkdir: 32769: Too many links
mkdir: 32770: Too many links
mkdir: 32771: Too many links
[...]

Which is 32768 - 2 for the '.' and '..' links. Trying to create too
many subdirectories just results in mkdir failing: the filesystem
itself is not damaged.

Cheers,

Matthew

[Wojciech Puchar]

Post by Matthew Seaman
With the implementation of UFS_DIRHASH the practical limit on the
size of directories is now a great deal larger. In particular
the slow down caused by linear search through the contents has been

but - try making (by shell script for example) empty files.

it creates it fast and rapidly slows at about 320000 on my machine.

i have UFS_DIRHASH

[Ian]

Post by Matthew Seaman

Post by Jeremy Chadwick
I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.

With the implementation of UFS_DIRHASH the practical limit on the
size of directories is now a great deal larger. In particular
the slow down caused by linear search through the contents has been
eliminated. See ffs(7). 10,000 files or sub-directories, whist
not a particularly elegant setup, is actually not unworkable
nowadays.

Well that's certainly been my experience so far. Still, I now know we will run
into problems when we hit the 32,768 limit, so I'll start designing something
better.

Cheers,

[Dan]

Post by Jeremy Chadwick
I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.

On a related note there are filesystems that handle many
files/directories very quickly. They use alternate tree data structures
that behave quicker. ReiserFS is one of them. I believe XFS does quite
well too. FreeBSD should have adapted XFS in addition to ZFS. ZFS is a
resource monster. Shame, really. XFS is freely available in Linux for a
number of years. Hammer, the new FS for FreeBSDs is available for
DragonflyBSD.

[Jeremy Chadwick]

Post by Dan

Post by Jeremy Chadwick
I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.

On a related note there are filesystems that handle many
files/directories very quickly. They use alternate tree data structures
that behave quicker. ReiserFS is one of them. I believe XFS does quite
well too. FreeBSD should have adapted XFS in addition to ZFS. ZFS is a
resource monster. Shame, really. XFS is freely available in Linux for a
number of years. Hammer, the new FS for FreeBSDs is available for
DragonflyBSD.

If you really think HAMMER accomplishes the same goals as ZFS, you are
sadly mistaken.

I have no idea about XFS.

[Modulok]

Post by Erik Trulsson
Personally I cannot think of any situation where one would actually want

(let alone need) as many as 30000 or more subdirectories in a single
directory.

"No one will ever need more than 640K of memory!"

Be careful.
-Modulok-

[Erik Trulsson]

Post by Erik Trulsson

Post by Erik Trulsson
Personally I cannot think of any situation where one would actually want

(let alone need) as many as 30000 or more subdirectories in a single
directory.
"No one will ever need more than 640K of memory!"

Not quite the same thing. One major problem with having lots of entries in
a directory is for humans using it (who have not become significantly
faster or more capable over the recent decades.) Having lots of entries in
a single directory is simply very unwieldy. There are is a reason why
people invented hierarchichal files systems with directories and
sub-directories, you know.

For those situations where the directory is not intended to be looked at by
a human, but only by programs, then there are more efficient ways of storing
the data if you need that many entries. (A real database system, for
example.)


Besides, most (all?) of the situations where one might concievably want many
entries in a single directory, what one would usually want is lots of files,
not lots of sub-directories - once you start using sub-directories, you
might as well use more than a single level of them.

[Wojciech Puchar]

Post by Jeremy Chadwick
If you really think HAMMER accomplishes the same goals as ZFS, you are
sadly mistaken.

it will be OK to achieve the goals it is advertised to achieve.

[Wojciech Puchar]

Post by Dan
number of years. Hammer, the new FS for FreeBSDs is available for
DragonflyBSD.

i would like to see final (now still beta) version of hammer in action.
it's ADVERTISED features are great. but ZFS features was (and are)
ADVERTISED great too while we see the result.

Hammer would be great if it will be as advertised :)

[Wojciech Puchar]

Post by n***@people.net.au
Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a single
directory.
This number will keep rising and I assume UFS2 has a limit to the number of
sub-directories in a single directory - can anyone tell me what it is?

make sure your kernel is compiled with

options UFS_DIRHASH


or it will be slow


the limit is 32765, just because link count is 2 bytes wide and each
subdir adds two to base directory. you have to change to 2 level
hierarchy.


with files - i started creating empty files, it turned slow after
about 320000.

[Robert Huff]

Post by Wojciech Puchar
the limit is 32765, just because link count is 2 bytes wide and
each subdir adds two to base directory. you have to change to 2
level hierarchy.

Question (for anyone who has an informed opinion):
If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?


Robert Huff

[Jeremy Chadwick]

Post by Robert Huff

Post by Wojciech Puchar
the limit is 32765, just because link count is 2 bytes wide and
each subdir adds two to base directory. you have to change to 2
level hierarchy.

If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

At this point, I think this topic should be relocated to the freebsd-fs
mailing list.

[Erik Trulsson]

Post by Robert Huff

Post by Wojciech Puchar
the limit is 32765, just because link count is 2 bytes wide and
each subdir adds two to base directory. you have to change to 2
level hierarchy.

If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

It probably could be expanded to 32 bits if that was deemed useful.
Doing that would of course require re-creating any existing filesystems
since the on-disk format would change, which would be a PITA for users,
but certainly possible.

It is rare that anybody actually encounter this limit however. I would
even say that if you have more than a couple of thousand entries in a single
directory, then you are probably doing something wrong.

Personally I cannot think of any situation where one would actually want
(let alone need) as many as 30000 or more subdirectories in a single
directory.

[Dan]

Post by Erik Trulsson
Personally I cannot think of any situation where one would actually want
(let alone need) as many as 30000 or more subdirectories in a single
directory.

I've seen some Java apps that use the FS as the DB. Nothing wrong with
that. I think an FS can be quite a good DB, if implemented well. This gives
many data manipulation options with traditional FS tools, shell
scripts, etc.

Large Maildirs for postfix and qmail/Courier. Some people don't delete
email at all.

[Erik Trulsson]

Post by Dan

Post by Erik Trulsson
Personally I cannot think of any situation where one would actually want
(let alone need) as many as 30000 or more subdirectories in a single
directory.

I've seen some Java apps that use the FS as the DB. Nothing wrong with
that. I think an FS can be quite a good DB, if implemented well. This gives
many data manipulation options with traditional FS tools, shell
scripts, etc.

Lets just say that there are reasons why the major database systems
generally use their own methods to store and organize the data rather
than rely on the file system for that.

Besides, for most database applications I can think of, what you would
need are lots of *files*, which do not have any special limitations other
than the the total space and number of i-nodes on the filesystem.
Even if you were using the FS as a DB I can't think of any good reason
to need 30000+ subdirectories in a single directory.

Post by Dan
Large Maildirs for postfix and qmail/Courier. Some people don't delete
email at all.

Again, that requires lots of files, not lots of subdirectories.

[d***@ourbrains.org]

Post by Erik Trulsson
Besides, for most database applications I can think of, what you would
need are lots of *files*, which do not have any special limitations other
than the the total space and number of i-nodes on the filesystem.
Even if you were using the FS as a DB I can't think of any good reason
to need 30000+ subdirectories in a single directory.

Post by Dan
Large Maildirs for postfix and qmail/Courier. Some people don't delete
email at all.

Again, that requires lots of files, not lots of subdirectories.

I agree, I misread the OP.

[Robert Huff]

Post by Erik Trulsson

Post by Robert Huff
If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

It probably could be expanded to 32 bits if that was deemed
useful. Doing that would of course require re-creating any
existing filesystems since the on-disk format would change, which
would be a PITA for users, but certainly possible.

I seem to remember at least one case (3.x -> 4.0 ????) where a
major version change had no upgrade path - to get the new stuff you
had to reinstall.
But I agree there's no reason based on current evidence to do
this.
Thanks.


Robert Huff

[Erik Trulsson]

Post by Robert Huff

Post by Erik Trulsson

Post by Robert Huff
If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

It probably could be expanded to 32 bits if that was deemed
useful. Doing that would of course require re-creating any
existing filesystems since the on-disk format would change, which
would be a PITA for users, but certainly possible.

I seem to remember at least one case (3.x -> 4.0 ????) where a
major version change had no upgrade path - to get the new stuff you
had to reinstall.

You are probably thinking of the 4.x -> 5.x upgrade where you pretty much
had to reinstall if you wanted to switch from UFS1 to UFS2. (But you could
of course keep using UFS1 if you wanted.)

Post by Robert Huff
But I agree there's no reason based on current evidence to do
this.
Thanks.
Robert Huff

[matt donovan]

Post by Erik Trulsson

Post by Robert Huff

Post by Erik Trulsson

Post by Robert Huff
If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

It probably could be expanded to 32 bits if that was deemed
useful. Doing that would of course require re-creating any
existing filesystems since the on-disk format would change, which
would be a PITA for users, but certainly possible.

I seem to remember at least one case (3.x -> 4.0 ????) where a
major version change had no upgrade path - to get the new stuff you
had to reinstall.

You are probably thinking of the 4.x -> 5.x upgrade where you pretty much
had to reinstall if you wanted to switch from UFS1 to UFS2. (But you could
of course keep using UFS1 if you wanted.)

Post by Robert Huff
But I agree there's no reason based on current evidence to do
this.
Thanks.
Robert Huff

--
<Insert your favourite quote here.>
Erik Trulsson
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "

Kind of hard to get XFS in freeBSD with it being a "dead" filesystem that is
no longer being developed, probably to port it it would need a lot of code
changes.

[Wojciech Puchar]

Post by Robert Huff

Post by Wojciech Puchar
the limit is 32765, just because link count is 2 bytes wide and
each subdir adds two to base directory. you have to change to 2
level hierarchy.

If there any technical reason that couldn't be expanded to 32
bits? Or is it possible but not done for historical or
policy reasons, and if so what are they?

looking at /usr/include/ufs/ufs/dinode.h - i see

int64_t di_spare[3];


and i have really no idea why time uses as much as 8+4 bytes like that:

ufs_time_t di_mtime; /* 40: Last modified time. */
int32_t di_mtimensec; /* 64: Last modified time. */


i think it is not a problem to make link count 32-bit, and - why "spare"
space are not just used for more direct/indirect blocks