Discussion:
How to connect a jail to the web ?
(too old to reply)
Brice ERRANDONEA
2010-08-10 11:01:24 UTC
Permalink
Hello,

I've just created my first FreeBSD jail in order to install a web server inside.
But I don't know how to connect it to the web. When I try pinging a http
website, it doesn't work. Of course, it works when I do it from outside the
jail.

Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.

Here's how I built and started my jail. I had already run make buildworld when
upgrading to 8.1 release :

# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh

I guess this must be a very basic question but please help me.
Julien Cigar
2010-08-10 11:32:15 UTC
Permalink
Post by Brice ERRANDONEA
Hello,
I've just created my first FreeBSD jail in order to install a web server inside.
But I don't know how to connect it to the web. When I try pinging a http
website, it doesn't work. Of course, it works when I do it from outside the
jail.
Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.
Here's how I built and started my jail. I had already run make buildworld when
# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh
I guess this must be a very basic question but please help me.
make sure NAT is enabled on the host..
I use PF for that with something like (/etc/pf.conf):

ext_if="bce0"
int_if="bce1"
internal_net="192.168.0.0/24"
nat on $ext_if from $internal_net to any -> ($ext_if)
Post by Brice ERRANDONEA
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
--
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.
Valentin Bud
2010-08-10 11:42:27 UTC
Permalink
Post by Brice ERRANDONEA
Hello,
I've just created my first FreeBSD jail in order to install a web server inside.
But I don't know how to connect it to the web. When I try pinging a http
website, it doesn't work. Of course, it works when I do it from outside the
jail.
Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.
Here's how I built and started my jail. I had already run make buildworld when
# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh
I guess this must be a very basic question but please help me.
Hello,

To be able to ping from inside the jail you need raw sockets
activated on the host.

sysctl security.jail.allow_raw_sockets=1

For ease of configuration you could use ezjail - a jail administration
framework written
in shell or if you plan to use lots of jails (20+) you could try qjail which
is also a jail
administration framework.

have a great day,
v
--
network warrior
Roland Smith
2010-08-10 13:08:34 UTC
Permalink
Post by Brice ERRANDONEA
Hello,
I've just created my first FreeBSD jail in order to install a web server
inside. But I don't know how to connect it to the web. When I try pinging a
http website, it doesn't work. Of course, it works when I do it from outside
the jail.
There are a couple of things you need to keep in mind.

- The IP address you're using for a jail is usually an alias for an existing
interface. I think this is done to make routing easier. My system is
configured as a gateway, and I've aliased the IP adresses for my jails to
the interaface of the internal trusted network.
- You should really use the rc interface for starting jails; it's much easier.
Post by Brice ERRANDONEA
Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.
See below.
Post by Brice ERRANDONEA
Here's how I built and started my jail. I had already run make buildworld when
# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
Do not forget to create an empty /etc/fstab in your jail;

# touch /usr/prison/etc/fstab

You'll also need to create an appropriate /etc/rc.conf file in the jail. The
following should be a starting point;

devfs_system_ruleset="devfsrules_jail"
network_interfaces=""
sshd_enable="YES"
sendmail_enable="NO"
rpcbind_enable="NO"
Post by Brice ERRANDONEA
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh
You should use the full path name of the program you want to run.

# jail /usr/prison ServeurWeb 192.1.1.1 /bin/csh

If you want to start the rc system in the jail;

# jail /usr/prison ServeurWeb 192.1.1.1 /bin/sh /etc/rc

I've detailed my setpup on a webpage. Maybe it will be of use to you;

http://www.xs4all.nl/~rsmith/unix/misc.xhtml#creatingavirtualserveronfreebsdwithajail8

Roland
--
R.F.Smith http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
Brice ERRANDONEA
2010-08-11 15:35:17 UTC
Permalink
I tried all of this without any result. But I won't give up.

What I want is a jail with an Apache http server running inside. So, the jail
must have a public IPv4 and access to the web.

What I'd understood of the jails' role (but I must have misunderstood) is that
it will have a different public ip than the host, so that if a pirate manage to
crack the server, he will only have access to the jail (the real public ip of
the host remaining secret). Then I'm surprised to learn that such traffic will
be routed through the host.

The jail is created. The next step now is to install the ports collection inside
with portsnap fetch. But each time I try to run this command inside the jail
(with jexec), I get the same answer :

Looking up portsnap.FreeBSD.org mirrors... none found.
Fetching public key from portsnap.FreeBSD.org... failed.
No mirrors remaining, giving up.

This makes me think my jail is not connected to the web. To check this, I tried
to ping various know websites. When I tried domain names, like "ping
www.freebsd.org", this error message appears :

ping: cannot resolve www.freebsd.org : Host name lookup failure

So, I can't contact DNS servers able to translate www.freebsd.org to its ip.
Since I know this ip, I tried : "ping 69.147.83.33". This time, the error
message is :

ping: socket: Operation not permitted

From this, I concluded my jail was not connected to the web. Meanwhile, I've
understood that, anyway, the ping command is forbidden inside a jail. But the
"portsnap fetch" one is not.

It seems that the local ip given to the jail has to be an alias of an existing
one. I'm not on a local network so I only have 2 real network interfaces : rl0
(192.168.1.38) and the loopack lo0 (127.0.0.1).

192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail. By the way, I
wonder which one I will be able to choose if I ever have to create a second
jail. And also how the computer knows which data is for the jail and which one
is for the loopback.

I also added the line "net.inet.ip.forwarding=1" to sysctl.conf (on the host).
And here is the rc.conf of my jail :

devfs_system_ruleset="devfsrules_jail"
network_interfaces=""
sshd_enable="YES"
sendmail_enable="NO"
rpcbind_enable="NO"

Despite the sshd_enable="YES" line, I can't ssh from the host to the jail. Well,
I can... The first time I did it, I was asked if I wanted to add the jail to the
list of known hosts. I did it. No problem there. But, immediatly after that,
instead of displaying "login :", the system displayed "passwd :". And none of
the passwords I had set with sysinstall (for the root and the common user) were
accepted. That's why I can only run commands inside the jail running jexec. It's
not that big problem for the moment but one purpose of the jail is also (I
believe) to ssh into them from a distant computer without accessing to the host.

It was not clear after the various answers I received if I had to use a firewall
or not so I tried both ways.

Without the firewall, the rc.conf of my host is :

hostname="FreeBSD.ici"
ifconfig_rl0="DHCP"
keymap="fr.iso.acc" (yes, I'm french)
moused_enable="YES"
saver="dragon"
hald_enable="YES"
dbus_enable="YES"
devfs_system_ruleset="localrules"

jail_enable="NO"
jail_list="MaPrison"
jail_interface="lo0" (I also tried rl0 here)
jail_devfs_ruleset="devfsrules_jail"
jail_devfs_enable="YES"

jail_server_rootdir="/usr/prison"
jail_server_hostname="MaPrison"
jail_server_ip="127.0.0.1"

gateway_enable="YES"
router_enable="YES"

Since I've added this last line (router_enable="YES"), I have to press Enter at
the end of the bootup process to obtain the "login :". Again, it's not a big
problem but nonetheless a strange one.

With this configuration, portsnap fetch continues to give me the same error
message I told before.

With the firewall (pf), now, the rc.conf of my host becomes :

hostname="FreeBSD.ici"
ifconfig_rl0="DHCP"
keymap="fr.iso.acc"
moused_enable="YES"
saver="dragon"
hald_enable="YES"
dbus_enable="YES"
devfs_system_ruleset="localrules"

jail_enable="NO"
jail_list="MaPrison"
jail_interface="lo0"
jail_devfs_ruleset="devfsrules_jail"
jail_devfs_enable="YES"

jail_server_rootdir="/usr/prison"
jail_server_hostname="MaPrison"
jail_server_ip="127.0.0.1"

gateway_enable="YES"
pf_enable="YES"
pf_rules="/etc/pf.conf"
pflog_enable="YES"
pflog_logfile="/var/log/pflog"

And here's the /etc/pf.conf :

ext_if="rl0"
int_if="rl0"

Same result for portsnap fetch.


A lot of questions, isn't it. I guess I must have made a lot of mistakes. But I
can't believe I'm the first one who tries to install a web server in a jail.
This must be a well known process.

Thanks to those who helped me and to those who will !

Good evening

Brice




________________________________
De : Roland Smith <***@xs4all.nl>
À : Brice ERRANDONEA <***@yahoo.fr>
Envoyé le : Mer 11 août 2010, 13h 23min 34s
Objet : Re: Re : Re : How to connect a jail to the web ?
OK, I'll try this. And, as you suggested, I switch my jail's IP to
192.168.1.1. Why do you use age0 as ext_if and not rl0 ?
Because rl(4) is just not the best quality network chip. It's really windows
quality hardware. The age(4) is on the motherboard, and I couldn't find a
fxp(4) or em(4) based network card.
Here's my ifconfig. Which interfaces should I use for ext_if in pf.conf ?
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
In your case, the above rl0 is the only _real_ network chip. As you can see
from the "UP" flag, only rl0 and lo0 are actually active (and the loopback
interface is always there). They also are the only ones that have an actual IP
address.

If you don't want to run a firewall, you can alternatively add
'router_enable="YES"' to /etc/rc.conf. This will start the routed(8) daemon
which by default forwards packets between interfaces.
fwe0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:11:06:99:8a:ff
ch 1 dma -1
fwip0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lladdr 0.11.6.66.0.99.8a.ff.a.2.ff.fe.0.0.0.0
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
You could alias your jail to lo0.

Roland
--
R.F.Smith http://www.xs4all.nl/~rsmith/
[plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated]
pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
Oliver Fromme
2010-08-11 16:46:20 UTC
Permalink
Post by Brice ERRANDONEA
I tried all of this without any result. But I won't give up.
What I want is a jail with an Apache http server running inside.
So, the jail must have a public IPv4 and access to the web.
Not necessarily. Of course, the jail _can_ have a public
IP address. This will make things easier.

But some people prefer to give their jails private addresses
or even aliases on lo0 (e.g. 127.0.0.2). In order to access
such a jail from the outside, the host has to forward packets
from and to the private address. This can be done with IPFW
"fwd" rules, for example.
Post by Brice ERRANDONEA
What I'd understood of the jails' role (but I must have
misunderstood) is that it will have a different public ip than the
host, so that if a pirate manage to crack the server, he will only
have access to the jail (the real public ip of the host remaining
secret).
Yes, it has advantages to give a jail its own IP address,
but it's not strictly necessary. The IP address can be
shared with the host and with other IP addresses if you
prefer.

It's also possible to give the jail the host's IP address
during installation, so things like portsnap, pkg_add -r
and similar will run without trouble, and then switch the
jail to its final IP address.
Post by Brice ERRANDONEA
Then I'm surprised to learn that such traffic will be routed
through the host.
Routing happens globally (unless you use VIMAGE and/or
multiple FIBs, but let's forget about these for now
because they make things even more complicated, and
you probably don't need them). By default there is only
one routing table inside the kernel, through which all
packets go. So, packets from your jails go through the
same routing table as packets from yur host.
Post by Brice ERRANDONEA
The jail is created. The next step now is to install the ports
collection inside with portsnap fetch. But each time I try to run
Looking up portsnap.FreeBSD.org mirrors... none found. Fetching
public key from portsnap.FreeBSD.org... failed. No mirrors
remaining, giving up.
This makes me think my jail is not connected to the web.
This has nothing to do with the web. Maybe you confuse
web and internet or network?

Obviously your jail cannot do DNS lookups, i.e. it cannot
resolve host names.
Post by Brice ERRANDONEA
So, I can't contact DNS servers able to translate www.freebsd.org to
its ip. Since I know this ip, I tried : "ping 69.147.83.33". This
ping: socket: Operation not permitted
ping(1) uses raw sockets in order to be able to send and
receive ICMP packets. By default, raw sopckets or disallowed
in jails. To change that, use this command on the host:

sysctl security.jail.allow_raw_sockets=1

Add an entry to /etc/sysctl.conf so the setting will survive
reboots.
Post by Brice ERRANDONEA
It seems that the local ip given to the jail has to be an alias
of an existing one.
No, it must simply be an existing address, i.e. it must be
configured on one of your interfaces (whether alias or not).
Post by Brice ERRANDONEA
I'm not on a local network so I only have 2
real network interfaces : rl0 (192.168.1.38) and the loopack lo0
(127.0.0.1).
So you can use one of those two addresses, or you can add
aliases (e.g. 192.168.1.39) and then use that one.

Of course you can only use addresses that you "own" and
that will work on your network. If addresses are assigned
to you by an ISP or administrator, then you can only use
those.
Post by Brice ERRANDONEA
192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail.
Well, localnet addresses are not routed. If you give your
jail a localnet address, it won't be able to access the
network outside of the host. (Unless you take measures
to rewrite/translate the addresses and forward them.)
That's why DNS and portsnap don't work.

I suggest using the address 192.168.1.38 for the jail,
at least during installation. Make sure that the file
/etc/resolv.conf inside the jail is correct, so DNS will
work. Copying it from the host should be sufficient.

By the way, you don't have to build ports inside the jail.
Of course you *can* do that, but there are other ways, too.
For example, you could build packages (apache etc.) on
the host, or in a different jail, or even on a different
machine, and then use pkg_add(8) inside your jail to
install them.
Post by Brice ERRANDONEA
By the way, I wonder which one I will be able to choose if I ever
have to create a second jail.
Multiple jails can share the same address if required.
Post by Brice ERRANDONEA
And also how the computer knows which data is for the jail and which
one is for the loopback.
Services (such as apache) listen on certain ports for
connections. For example, the default port for the HTTP
protocol is 80. So, when someone is trying to open a
connection to your IP address on port 80, your kernel
looks it up in its table of listening TCP sockets and
find the apache process which is running inside the jail.
So the connection is handed to the jail.

(This is a bit oversimplifying, but basically that's how
it works.)
Post by Brice ERRANDONEA
I also added the line "net.inet.ip.forwarding=1" to sysctl.conf (on the host).
You don't need that one. It's only required when your
machine should act as a router, i.e. forward packets to
other hosts.
Post by Brice ERRANDONEA
Despite the sshd_enable="YES" line, I can't ssh from the host to the
jail. Well, I can... The first time I did it, I was asked if I wanted
to add the jail to the list of known hosts. I did it. No problem
there. But, immediatly after that, instead of displaying "login :",
the system displayed "passwd :".
That's normal. ssh never asks for the login. You can use the -l
option if you need to specify a different user name (or put it in your
~/.ssh/config).
Post by Brice ERRANDONEA
And none of the passwords I had set with sysinstall (for the root and
the common user) were accepted.
Are you sure that those passwords are set *inside* the jail?
You can go into the jail with jexec (or even chroot) and
then set a new password.
Post by Brice ERRANDONEA
It's not that big problem for the moment but one purpose of the jail
is also (I believe) to ssh into them from a distant computer without
accessing to the host.
That's not a good idea. ssh access should not be open to
the public. It's better to log into the host first, then
log into the jail from there.

Some paranoid people have a special "login jail". They
ssh into the login jail, then log into the host or into
other jails from there. The host accepts ssh only from
localhost. But please forget this immediately; we don't
want to make things more complicated than necessary.
Post by Brice ERRANDONEA
It was not clear after the various answers I received if I had to use
a firewall or not so I tried both ways.
If your just starting with jails, it's better not to use
a firewall for the jail. First get the jail running.
When it's running, you can think about adding firewall
rules to make it more secure.

A firewall is *not* required to get jails working.
Post by Brice ERRANDONEA
gateway_enable="YES"
router_enable="YES"
Remove both. You don't need either of those.

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"Above all, they contribute to the genetic diversity in the
operating system pool. Which is a good thing."
-- Ruben van Staveren, on the question which BSD OS is the best one.
Brice ERRANDONEA
2010-08-11 18:14:04 UTC
Permalink
Thank you very much for your answer. It helped me understand some elements. But
portsnap still doesn't work.
Post by Oliver Fromme
Post by Brice ERRANDONEA
So, I can't contact DNS servers able to translate www.freebsd.org to
its ip. Since I know this ip, I tried : "ping 69.147.83.33". This
ping: socket: Operation not permitted
ping(1) uses raw sockets in order to be able to send and
receive ICMP packets. By default, raw sopckets or disallowed
sysctl security.jail.allow_raw_sockets=1
Add an entry to /etc/sysctl.conf so the setting will survive
reboots.
I did it but ping still doesn't work.
Post by Oliver Fromme
Post by Brice ERRANDONEA
192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail.
Well, localnet addresses are not routed. If you give your
jail a localnet address, it won't be able to access the
network outside of the host. (Unless you take measures
to rewrite/translate the addresses and forward them.)
That's why DNS and portsnap don't work.
I suggest using the address 192.168.1.38 for the jail,
at least during installation. Make sure that the file
/etc/resolv.conf inside the jail is correct, so DNS will
work. Copying it from the host should be sufficient.
Isn't 192.168.1.38 a localnet address too ? Do you mean I should use the public
ip of my computer here ?
Post by Oliver Fromme
By the way, you don't have to build ports inside the jail.
Of course you *can* do that, but there are other ways, too.
For example, you could build packages (apache etc.) on
the host, or in a different jail, or even on a different
machine, and then use pkg_add(8) inside your jail to
install them.
I prefer doing that way. I will use apache later so I will have to connect the
jail to internet anyway.
Post by Oliver Fromme
Post by Brice ERRANDONEA
And also how the computer knows which data is for the jail and which
one is for the loopback.
Services (such as apache) listen on certain ports for
connections. For example, the default port for the HTTP
protocol is 80. So, when someone is trying to open a
connection to your IP address on port 80, your kernel
looks it up in its table of listening TCP sockets and
find the apache process which is running inside the jail.
So the connection is handed to the jail.
(This is a bit oversimplifying, but basically that's how
it works.)
OK. This is clear. And it explains how multiple jails can share the same
address.
Post by Oliver Fromme
Post by Brice ERRANDONEA
Despite the sshd_enable="YES" line, I can't ssh from the host to the
jail. Well, I can... The first time I did it, I was asked if I wanted
to add the jail to the list of known hosts. I did it. No problem
there. But, immediatly after that, instead of displaying "login :",
the system displayed "passwd :".
That's normal. ssh never asks for the login. You can use the -l
option if you need to specify a different user name (or put it in your
~/.ssh/config).
Of course. I'm loosing my mind with all that jail trouble. It works perfectly
well with le -l option.
Post by Oliver Fromme
Some paranoid people have a special "login jail". They
ssh into the login jail, then log into the host or into
other jails from there. The host accepts ssh only from
localhost. But please forget this immediately; we don't
want to make things more complicated than necessary.
I thought it was intended to be impossible to access the host from the jail. But
you're right : I'll forget that.

So, we're progressing. But the problem is not over yet. Any other idea ?

Have a good evening, anyway.

Brice
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"Above all, they contribute to the genetic diversity in the
operating system pool. Which is a good thing."
-- Ruben van Staveren, on the question which BSD OS is the best one.
Jack Raats
2010-08-11 17:30:08 UTC
Permalink
It seems that you have DNS problems.
Login in your jail
go to /etc

Make a file called resolv.conf
which contains:

domain your_jail_domain
nameserver your_namerserver

and it will work...

Jack

PS sorry for the top posting.
I'm using outlook express :-(






----- Original Message -----
From: "Brice ERRANDONEA" <***@yahoo.fr>
To: "Roland Smith" <***@xs4all.nl>; <freebsd-***@freebsd.org>
Sent: Wednesday, August 11, 2010 5:35 PM
Subject: Re : How to connect a jail to the web ?


I tried all of this without any result. But I won't give up.

What I want is a jail with an Apache http server running inside. So, the
jail
must have a public IPv4 and access to the web.

What I'd understood of the jails' role (but I must have misunderstood) is
that
it will have a different public ip than the host, so that if a pirate manage
to
crack the server, he will only have access to the jail (the real public ip
of
the host remaining secret). Then I'm surprised to learn that such traffic
will
be routed through the host.

The jail is created. The next step now is to install the ports collection
inside
with portsnap fetch. But each time I try to run this command inside the jail
(with jexec), I get the same answer :

Looking up portsnap.FreeBSD.org mirrors... none found.
Fetching public key from portsnap.FreeBSD.org... failed.
No mirrors remaining, giving up.

This makes me think my jail is not connected to the web. To check this, I
tried
to ping various know websites. When I tried domain names, like "ping
www.freebsd.org", this error message appears :

ping: cannot resolve www.freebsd.org : Host name lookup failure

So, I can't contact DNS servers able to translate www.freebsd.org to its ip.
Since I know this ip, I tried : "ping 69.147.83.33". This time, the error
message is :
Rocky Borg
2010-08-11 20:05:03 UTC
Permalink
Post by Brice ERRANDONEA
I tried all of this without any result. But I won't give up.
What I want is a jail with an Apache http server running inside. So, the jail
must have a public IPv4 and access to the web.
I've been in the same boat as you and there isn't a lot of clear
documentation that works in all situations. After reading tons of stuff
on the subject I finally figured out what should work in almost every
situation. Rather than fit everything in an email I put together a HOWTO
on the freebsd forums. This should get you up and running quickly and if
you have any problems or questions don't hesitate to ask.

http://forums.freebsd.org/showthread.php?t=16860
Rocky Borg
2010-08-10 13:54:14 UTC
Permalink
Post by Brice ERRANDONEA
Hello,
I've just created my first FreeBSD jail in order to install a web server inside.
But I don't know how to connect it to the web. When I try pinging a http
website, it doesn't work. Of course, it works when I do it from outside the
jail.
Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.
Here's how I built and started my jail. I had already run make buildworld when
# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh
I guess this must be a very basic question but please help me.
I would highly recommend ezjail for setting up jails. Although you
should still read the handbook on jails so you understand the overall
mechanics. Reading ezjails man page makes it very easy to setup and
deploy new jails in the future. The only thing you need to do inside a
jail setup with ezjail to connect to the web is put nameservers in
/etc/resolv.conf

For setting it up on your host system you can do something like this
(there are a couple of ways you can do it, I've just found this to be
the most portable).

host rc.conf
#Put jail on loopback device
cloned_interfaces="lo1"
ifconfig_lo1="inet 10.1.1.1 netmask 255.255.255.0"

# Enable port forwarding and packet filtering
gateway_enable="YES"
pf_enable="YES"
pf_rules="/etc/pf.conf"

# Jails
ezjail_enable="YES"

host pf.conf, find your interface name via ifconfig
#INTERFACES
ext_if="em0"

# nat from jails to your network cards ip
nat on $ext_if from 10.1.1.0/24 to any -> XXX.XXX.XXX.XXX

Here are some resource I found helpful when I was setting up jails for
the first time. Be aware some ezjail tutorials are really old and you
should read the man page first as that is current.

http://www2.budzien.com/wiki/Wiki.jsp?page=UsingEzJail
http://wael.nasreddine.com/blog/jail-servers.html
http://www.jeroen.se/articles/freebsd_jail_laptop_dhcp.php
Fbsd8
2010-08-11 00:02:57 UTC
Permalink
Post by Brice ERRANDONEA
Hello,
I've just created my first FreeBSD jail in order to install a web server inside.
But I don't know how to connect it to the web. When I try pinging a http
website, it doesn't work. Of course, it works when I do it from outside the
jail.
Another problem, probably linked to the first one, I can't run rc within the
jail, even as the jail's root. It says : permission denied.
Here's how I built and started my jail. I had already run make buildworld when
# mkdir /usr/prison
# cd /usr/src
# make installworld DESTDIR=/usr/prison
# make distribution DESTDIR=/usr/prison
# mount -t devfs devfs /usr/prison/dev
# jail -c path=/usr/prison host.hostname=ServeurWeb ip4.addr=192.1.1.1 persist
# jail /usr/prison ServeurWeb 192.1.1.1 csh
I guess this must be a very basic question but please help me.
1. ping is a security risk from within a jail and is disabled by design.
(read jail(8) for details). No use using a jail if the first thing you
do is re-enable ping in the jail. To test for public internet connection
from within a jail use dig or whois commands.

2. Using the hosts firewall to drive traffic to a jail is a sign you
have your jail incorrectly configured or do not understand how jails are
intended to work.

3. Jail do not have a network stack of their own, so they cant have a
firewall. The host's firewall and and network stack are in control.

4. There are 2 utilities for creating jails. Qjail the better documented
of the 2, is designed for the novice which clearly you are. I strongly
suggest you checkout
http://sourceforge.net/projects/qjail
Randal L. Schwartz
2010-08-11 00:55:19 UTC
Permalink
Fbsd8> 2. Using the hosts firewall to drive traffic to a jail is a sign
Fbsd8> you have your jail incorrectly configured or do not understand
Fbsd8> how jails are intended to work.

OK, I'll bite. I thought this was the only way to do this. Can you
elaborate? I'll even accept URL pointers to go read. :)
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Fbsd8
2010-08-11 01:56:38 UTC
Permalink
Post by Randal L. Schwartz
Fbsd8> 2. Using the hosts firewall to drive traffic to a jail is a sign
Fbsd8> you have your jail incorrectly configured or do not understand
Fbsd8> how jails are intended to work.
OK, I'll bite. I thought this was the only way to do this. Can you
elaborate? I'll even accept URL pointers to go read. :)
ifconfig alias

man 8 ifconfig
Randal L. Schwartz
2010-08-11 02:09:46 UTC
Permalink
Fbsd8> ifconfig alias

Fbsd8> man 8 ifconfig

Yup, and using that, I can give a private 10.x address to my jail.

How do I get it to face the public without a firewall rule?
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Fbsd8
2010-08-11 02:50:29 UTC
Permalink
Post by Randal L. Schwartz
Fbsd8> ifconfig alias
Fbsd8> man 8 ifconfig
Yup, and using that, I can give a private 10.x address to my jail.
How do I get it to face the public without a firewall rule?
No. Your jail is assigned it's ip address when you create it. The alias
gives the jail network access when you start the jail. Both ip address
must match.

Just assign the jail your public ip address when you create it.

"face the public" is a very large subject, which the answer depends on
your hardware configuration, registered domain names and static ip
addresses.

Using jails requires the host system administrator to be well trained in
networks and how public and private networks function. Jail
documentation is not going to teach you this.
Randal L. Schwartz
2010-08-11 02:59:39 UTC
Permalink
Fbsd8> No. Your jail is assigned it's ip address when you create it. The
Fbsd8> alias gives the jail network access when you start the jail. Both
Fbsd8> ip address must match.

Yup, and if that's a 10.x address, I'm not on the net. So I have to
route to it somehow.

Fbsd8> Just assign the jail your public ip address when you create it.

I was under the impression that the address had to be distinct, in order
to uniquely identify it. Are you saying that's not the case? If so,
the docs on jails are unclear.

Fbsd8> "face the public" is a very large subject, which the answer depends on your
Fbsd8> hardware configuration, registered domain names and static ip
Fbsd8> addresses.

Yes, I'm hoping not to burn a second or third public address for my
jail. Instead, I just want my jail to have a punch through (port 80,
port 25, etc) from my one public address. Is there a trick to this
without burning another public address? Or do I misunderstand (based on
poor docs) how a jail attaches itself to an interface?

Fbsd8> Using jails requires the host system administrator to be well
Fbsd8> trained in networks and how public and private networks
Fbsd8> function. Jail documentation is not going to teach you this.

Now you're just being condescending. It's fairly likely, almost
certain, that I've been dealing with IP traffic since before you could
type.

What I'm asking for is the specifics of Jails. I *know* how IP traffic
works, and even what alias does. What I don't know is FreeBSD's
particulars that make this either hard or easy. I *do* know about pf,
having administered an OpenBSD box for a number of years. I'm just new
to jails, and since you're the "expert", you might have a little
patience on that realm, please.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Fbsd8
2010-08-11 08:36:48 UTC
Permalink
Post by Randal L. Schwartz
Fbsd8> No. Your jail is assigned it's ip address when you create it. The
Fbsd8> alias gives the jail network access when you start the jail. Both
Fbsd8> ip address must match.
Yup, and if that's a 10.x address, I'm not on the net. So I have to
route to it somehow.
Fbsd8> Just assign the jail your public ip address when you create it.
I was under the impression that the address had to be distinct, in order
to uniquely identify it. Are you saying that's not the case? If so,
the docs on jails are unclear.
Fbsd8> "face the public" is a very large subject, which the answer depends on your
Fbsd8> hardware configuration, registered domain names and static ip
Fbsd8> addresses.
Yes, I'm hoping not to burn a second or third public address for my
jail. Instead, I just want my jail to have a punch through (port 80,
port 25, etc) from my one public address. Is there a trick to this
without burning another public address? Or do I misunderstand (based on
poor docs) how a jail attaches itself to an interface?
Fbsd8> Using jails requires the host system administrator to be well
Fbsd8> trained in networks and how public and private networks
Fbsd8> function. Jail documentation is not going to teach you this.
Now you're just being condescending. It's fairly likely, almost
certain, that I've been dealing with IP traffic since before you could
type.
What I'm asking for is the specifics of Jails. I *know* how IP traffic
works, and even what alias does. What I don't know is FreeBSD's
particulars that make this either hard or easy. I *do* know about pf,
having administered an OpenBSD box for a number of years. I'm just new
to jails, and since you're the "expert", you might have a little
patience on that realm, please.
First thing to keep in mind is jails were designed to be targeted by
unique public routable static ip address, in that configuration each
jail can run any mixture of services.

Different jails on the gateway host using the same public routable
static ip address can be targeted by service port number if that port
number is not in use on the host or any other jail. This is implied
usage,IE not specified in any control file.

Lets say the freebsd gateway host has a single static ip address and you
want jails on the gateway host to receive unsolicited inbound traffic
for web server (port 80) and mail server (port 25). Your domain name
points to the single static ip address. Create 2 jails assigned to the
single static ip address without the jail auto alias function enabled.
No gateway host firewall rules to stop inbound traffic on those ports,
or have those ports NATED, but should have statefull rules to let
traffic pass. The gateway host can not have a web server using port 80
or a mail server using port 25 or they will process the traffic before
the jails see it. The only service running on the web server jail is
apache listening on port 80 and the mail server jail (postfix) listening
on port 25. In this configuration the web server can even service
multiple domain name vhosts.

Now if the gateway host has a non-static ip address (dynamic ip address)
such as those assigned by ISP's providing DSL or cable internet services
your public ip address may change on you when the lease time expires or
the system reboots causing your jails to loose their public internet
access. Some domain name registers have function where you run a task on
you gateway host to monitor your public IP address, and if it changes
submits to your domain name register a automatic request to change the
ip address your domain name points to.

Another gotcha is some DSL or cable providers of public internet
services have their network designed as a LAN and you do not have a real
public routable ip address EVER. In this case your jails can only be
used for services restricted to your own private LAN. The service
provider is NATing your traffic at their front door. You are SOL.
Thomas Wahyudi
2010-08-11 06:51:35 UTC
Permalink
fbsd8> man 8 ifconfig
Yup, and using that, I can give a private 10.x address to my jail.
How do I get it to face the public without a firewall rule?
you need natd and firewall divert rule on jail host. Everything that
involve outside jail need must be configure at jail host level.
--
Thanks& Regards,

Thomas Wahyudi
Randal L. Schwartz
2010-08-11 13:29:59 UTC
Permalink
Thomas> On 11/08/2010 9:09, Randal L. Schwartz wrote:
fbsd8> man 8 ifconfig
Post by Randal L. Schwartz
Yup, and using that, I can give a private 10.x address to my jail.
How do I get it to face the public without a firewall rule?
Thomas> you need natd and firewall divert rule on jail host. Everything that involve
Thomas> outside jail need must be configure at jail host level.

Exactly as I suspected. Thanks for confirming it. I was just wondering
if fbsd8 was blowing smoke, and apparently, yes.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Matthew Seaman
2010-08-11 07:42:32 UTC
Permalink
Post by Randal L. Schwartz
Fbsd8> 2. Using the hosts firewall to drive traffic to a jail is a sign
Fbsd8> you have your jail incorrectly configured or do not understand
Fbsd8> how jails are intended to work.
OK, I'll bite. I thought this was the only way to do this. Can you
elaborate? I'll even accept URL pointers to go read. :)
Fbsd8's contention is ... contentious. Giving your jail an IP on the
loopback i/f, and then using NAT to redirect traffic for certain
selected ports lets you run services in the jail that need to bind to
some network address but that you never want exposed to the Internet.
Remember, unless you're using VIMAGE, jails don't have a loopback i/f of
their own. VIMAGE is cool, but as it's still incompatible with various
other kernel bits, I don't think it's quite ready for primetime yet.

Yes, you can achieve the same effect using firewall rules, but as I have
occasionally said before, firewalls should be optional -- ideally your
system should be secure even if you turn the firewall off.

Cheers,

Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: ***@infracaninophile.co.uk Kent, CT11 9PW
Randal L. Schwartz
2010-08-11 13:29:09 UTC
Permalink
Matthew> Yes, you can achieve the same effect using firewall rules, but
Matthew> as I have occasionally said before, firewalls should be
Matthew> optional -- ideally your system should be secure even if you
Matthew> turn the firewall off.

Well, I already have pf fired up to deal with web and ssh rate limiting,
so firing up a natd seems a bit redundant.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Matthew Seaman
2010-08-11 13:50:27 UTC
Permalink
Post by Randal L. Schwartz
Matthew> Yes, you can achieve the same effect using firewall rules, but
Matthew> as I have occasionally said before, firewalls should be
Matthew> optional -- ideally your system should be secure even if you
Matthew> turn the firewall off.
Well, I already have pf fired up to deal with web and ssh rate limiting,
so firing up a natd seems a bit redundant.
I meant that you could block access to private servers which need to
listen on public network ports by just using firewall rules, as opposed
to making the whole jail hang off a private interface and just
forwarding selected traffic to it.

For the second case, you would need pf to do the NAT'ing (or ipfw+natd
if that's your preference). With this trick of binding the sensitive
daemons to an address on the loopback, you are still secure even if pf
gets turned off. Of course, "secure" is not necessarily the same as
"working."

Cheers,

Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: ***@infracaninophile.co.uk Kent, CT11 9PW
David Allen
2010-08-11 14:10:06 UTC
Permalink
Post by Matthew Seaman
I meant that you could block access to private servers which need to
listen on public network ports by just using firewall rules, as opposed
to making the whole jail hang off a private interface and just
forwarding selected traffic to it.
For the second case, you would need pf to do the NAT'ing (or ipfw+natd
if that's your preference). With this trick of binding the sensitive
daemons to an address on the loopback, you are still secure even if pf
gets turned off. Of course, "secure" is not necessarily the same as
"working."
I've read comments in the past about setting up jails using local
loopback addresses, but I'm wondering if you wouldn't mind elaborating
on what the actual pf rules would look like.

Say you have 3 jails and more than one public IP address:

ns 127.0.0.2 public_ip_1
mail 127.0.0.3 public_ip_2
www 127.0.0.4 public_ip_3

You want to pass port 25 traffic to/from the 'mail' jail. But you also
need that jail to use the correct public_ip address. Is that possible
without using, for example, pf's binat?

Thanks.
Matthew Seaman
2010-08-11 20:18:09 UTC
Permalink
Post by David Allen
Post by Matthew Seaman
I meant that you could block access to private servers which need to
listen on public network ports by just using firewall rules, as opposed
to making the whole jail hang off a private interface and just
forwarding selected traffic to it.
For the second case, you would need pf to do the NAT'ing (or ipfw+natd
if that's your preference). With this trick of binding the sensitive
daemons to an address on the loopback, you are still secure even if pf
gets turned off. Of course, "secure" is not necessarily the same as
"working."
I've read comments in the past about setting up jails using local
loopback addresses, but I'm wondering if you wouldn't mind elaborating
on what the actual pf rules would look like.
ns 127.0.0.2 public_ip_1
mail 127.0.0.3 public_ip_2
www 127.0.0.4 public_ip_3
You want to pass port 25 traffic to/from the 'mail' jail. But you also
need that jail to use the correct public_ip address. Is that possible
without using, for example, pf's binat?
Thanks.
Sure. In the best Blue Peter tradition[*], here's one I prepared earlier:

http://lists.freebsd.org/pipermail/freebsd-questions/2008-March/171748.html

While that talks about redirecting a couple of TCP and one UDP service
into a single jailed host, I think it's pretty clear how to get from
there to having several different jails each with running a different
service.

Cheers,

Matthew

[*] It's a British thing. You have to have been bought up here to
understand.
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: ***@infracaninophile.co.uk Kent, CT11 9PW
Oliver Fromme
2010-08-12 08:39:53 UTC
Permalink
Post by David Allen
I've read comments in the past about setting up jails using local
loopback addresses, but I'm wondering if you wouldn't mind elaborating
on what the actual pf rules would look like.
ns 127.0.0.2 public_ip_1
mail 127.0.0.3 public_ip_2
www 127.0.0.4 public_ip_3
You want to pass port 25 traffic to/from the 'mail' jail. But you also
need that jail to use the correct public_ip address. Is that possible
without using, for example, pf's binat?
Just for completeness, this is a little "how-to" that
describes how you do it with IPFW. You do not have to
configure NAT. One single fwd rule is sufficient.
The following example works on FreeBSD 8.1.

In this example, I'll use port 42, the jail has address
127.0.0.2 on lo0, and nc (netcat) is used in place of a
real daemon. The real (external) address of the host
machine is 10.5.5.5.

HOST# is the prompt of the server machine that hosts the
jail, JAIL# is the prompt within that host machine's
jail, and CLIENT$ is the prompt of a separate physical
machine on the same network which is used for testing
purposes.

First add an alias IP to the lo0 (localnet) interface.

HOST# ifconfig lo0 inet 127.0.0.2/32 alias

In order to make that permament, you have to add an
alias line to /etc/rc.conf, of course:

ifconfig_lo0_alias0="inet 127.0.0.2/32"

Check the addresses:

HOST# ifconfig lo0 | grep -w inet
inet 127.0.0.1 netmask 0xff000000
inet 127.0.0.2 netmask 0xffffffff

Install the IPFW fwd rule:

HOST# ipfw add 1 fwd 127.0.0.2 tcp from any to 10.5.5.5 42
00001 fwd 127.0.0.2 tcp from any to 10.5.5.5 dst-port 42

To make that permanent, add these lines to /etc/rc.conf:

firewall_enable="YES"
firewall_type="/etc/ipfw.conf"

And create a file /etc/ipfw.conf containing these lines:

-f flush
add fwd 127.0.0.2 tcp from any to 10.5.5.5 42

Ok, now start the jail. For the sake of this example,
we simply re-use the host's installed base, i.e. the
jail's root path is "/". For a real jail you would
use the jail's root directory, of course.

HOST# jail / testjail 127.0.0.2 /bin/sh -E

Finally start a netcat (nc) process in the jail.
In a real jail, this would be an apache process on
port 80, a mail transfer agent on port 25, whatever.

JAIL# nc -ln 42

Now the netcat process is listening on port 42 inside
the jail on the localnet address 127.0.0.2. You can
verify that with sockstat(1) on the host:

HOST# sockstat | grep -w 42
root nc 1953 3 tcp4 127.0.0.2:42 *:*

You can now connect to that "service" from a different
system on the network, using the external IP address
of the host. The IPFW fwd rule reroutes the packets
destined for port 42 to the jail's localnet address.

CLIENT$ echo Hello world | nc 10.5.5.5 42

As a result, netcat will echo the string "Hello world"
in the jail, and the nc process will terminate.

Note: In order to be able to use IPFW fwd rules, you
should have these two lines in your kernel config:

options IPFIREWALL
options IPFIREWALL_FORWARD

If you don't intend to use IPFW for anything else than
fwd, you can also include the following line, so you
don't have to install any additional "allow" rules:

options IPFIREWALL_DEFAULT_TO_ACCEPT

That's especially useful if you want to use IPFW for
forwarding only, and use another software for actual
packet filtering (i.e. pf or ipf).

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"With sufficient thrust, pigs fly just fine. However, this
is not necessarily a good idea. It is hard to be sure where
they are going to land, and it could be dangerous sitting
under them as they fly overhead." -- RFC 1925
Rocky Borg
2010-08-11 01:07:32 UTC
Permalink
Post by Fbsd8
1. ping is a security risk from within a jail and is disabled by
design. (read jail(8) for details). No use using a jail if the first
thing you do is re-enable ping in the jail. To test for public
internet connection from within a jail use dig or whois commands.
There is a vast difference between testing a network connection and
leaving something in for live deployment. Tools like ping and traceroute
are for network diagnostics. You can easily run into a situation where
dig and whois don't work but ping/traceroute will in which case you
quickly realize hostnames aren't resolving in a jail (or you can find
out where exactly packets stopped at). Meanwhile the person using only
dig and whois might be spinning their wheels trying to fix problems that
aren't really problems. They might of created a jail and have everything
setup except they forgot to create an /etc/resolv.conf in the jail.
There is nothing wrong with allowing raw sockets to get up and running
and then changing it back (the jail man page states to use caution with
raw sockets not a blatant don't do it).
Post by Fbsd8
2. Using the hosts firewall to drive traffic to a jail is a sign you
have your jail incorrectly configured or do not understand how jails
are intended to work.
If you have jails assigned to non routable ip's (i.e. 10.0.0.2,
10.0.0.3) how else would you redirect traffic coming in from your hosts
ip:(http_port, dns_port, etc..) to the corresponding jail that handles
it. I've read a bunch of stuff on jails and unless I missed something
(which is totally possible) using a NAT that's part of a firewall seems
like pretty standard fare. How else would you go about it?
Post by Fbsd8
3. Jail do not have a network stack of their own, so they cant have a
firewall. The host's firewall and and network stack are in control.
The documentation is rather sparse since it's so new and I personally
haven't used it but FreeBSD 8 has VIMAGE (network stack virtualization).

http://wiki.freebsd.org/Image/VNETSamples
http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto
http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet
Post by Fbsd8
4. There are 2 utilities for creating jails. Qjail the better
documented of the 2, is designed for the novice which clearly you are.
I strongly suggest you checkout
http://sourceforge.net/projects/qjail
You should probably preface this by saying you're the author of Qjail
and have been actively promoting it in a few places including the fbsd
forums. Nothing wrong with that I guess, but I still haven't been able
to figure out how it's any different(better?) than ezjail(which has both
an excellent website and man page).
Fbsd8
2010-08-11 02:23:31 UTC
Permalink
Post by Rocky Borg
Post by Fbsd8
1. ping is a security risk from within a jail and is disabled by
design. (read jail(8) for details). No use using a jail if the first
thing you do is re-enable ping in the jail. To test for public
internet connection from within a jail use dig or whois commands.
There is a vast difference between testing a network connection and
leaving something in for live deployment. Tools like ping and traceroute
are for network diagnostics. You can easily run into a situation where
dig and whois don't work but ping/traceroute will in which case you
quickly realize hostnames aren't resolving in a jail (or you can find
out where exactly packets stopped at). Meanwhile the person using only
dig and whois might be spinning their wheels trying to fix problems that
aren't really problems. They might of created a jail and have everything
setup except they forgot to create an /etc/resolv.conf in the jail.
There is nothing wrong with allowing raw sockets to get up and running
and then changing it back (the jail man page states to use caution with
raw sockets not a blatant don't do it).
The key verbiage here is "and then changing it back". Giving advice
without also saying why its disabled or that you should disable it when
completed testing is giving the op the wrong info.
Post by Rocky Borg
Post by Fbsd8
2. Using the hosts firewall to drive traffic to a jail is a sign you
have your jail incorrectly configured or do not understand how jails
are intended to work.
If you have jails assigned to non routable ip's (i.e. 10.0.0.2,
10.0.0.3) how else would you redirect traffic coming in from your hosts
ip:(http_port, dns_port, etc..) to the corresponding jail that handles
it. I've read a bunch of stuff on jails and unless I missed something
(which is totally possible) using a NAT that's part of a firewall seems
like pretty standard fare. How else would you go about it?
man 8 ifconfig

alias option
Post by Rocky Borg
Post by Fbsd8
3. Jail do not have a network stack of their own, so they cant have a
firewall. The host's firewall and and network stack are in control.
The documentation is rather sparse since it's so new and I personally
haven't used it but FreeBSD 8 has VIMAGE (network stack virtualization).
http://wiki.freebsd.org/Image/VNETSamples
http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto
http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet
This is pretty much experimental and nothing a sane person would think
of using in production.

Maybe in 9.0 the bugs will be worked out. Just have to wait and see.
Post by Rocky Borg
Post by Fbsd8
4. There are 2 utilities for creating jails. Qjail the better
documented of the 2, is designed for the novice which clearly you are.
I strongly suggest you checkout
http://sourceforge.net/projects/qjail
You should probably preface this by saying you're the author of Qjail
and have been actively promoting it in a few places including the fbsd
forums. Nothing wrong with that I guess, but I still haven't been able
to figure out how it's any different(better?) than ezjail(which has both
an excellent website and man page).
If you had really read both ezjail and qjail man pages you would not be
making this statement. They are as different as night and day. Qjail is
written for the novice with examples and includes many functions missing
from ezjail. Like the auto alias function that has been part of the jail
command since day one.
Jonathan McKeown
2010-08-11 09:35:41 UTC
Permalink
Post by Rocky Borg
You should probably preface this by saying you're the author of Qjail
and have been actively promoting it in a few places including the fbsd
forums.
That's interesting, given that you're replying to Fbsd8
<***@a1poweruser.com>. The announcement of qjail came from Aiza
<***@comclark.com>.

No reason why someone shouldn't use two email accounts, I guess; but I must
admit I'd naively assumed fbsd8 was independently endorsing aiza's utility.
Brice ERRANDONEA
2010-08-11 18:24:53 UTC
Permalink
Thank you very much for your answer. It helped me understand some elements. But
portsnap still doesn't work.
Post by Oliver Fromme
Post by Brice ERRANDONEA
So, I can't contact DNS servers able to translate www.freebsd.org to
its ip. Since I know this ip, I tried : "ping 69.147.83.33". This
ping: socket: Operation not permitted
ping(1) uses raw sockets in order to be able to send and
receive ICMP packets. By default, raw sopckets or disallowed
sysctl security.jail.allow_raw_sockets=1
Add an entry to /etc/sysctl.conf so the setting will survive
reboots.
I did it but ping still doesn't work.
Post by Oliver Fromme
Post by Brice ERRANDONEA
192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail.
Well, localnet addresses are not routed. If you give your
jail a localnet address, it won't be able to access the
network outside of the host. (Unless you take measures
to rewrite/translate the addresses and forward them.)
That's why DNS and portsnap don't work.
I suggest using the address 192.168.1.38 for the jail,
at least during installation. Make sure that the file
/etc/resolv.conf inside the jail is correct, so DNS will
work. Copying it from the host should be sufficient.
Isn't 192.168.1.38 a localnet address too ? Do you mean I should use the public
ip of my computer here ?
Post by Oliver Fromme
By the way, you don't have to build ports inside the jail.
Of course you *can* do that, but there are other ways, too.
For example, you could build packages (apache etc.) on
the host, or in a different jail, or even on a different
machine, and then use pkg_add(8) inside your jail to
install them.
I prefer doing that way. I will use apache later so I will have to connect the
jail to internet anyway.
Post by Oliver Fromme
Post by Brice ERRANDONEA
And also how the computer knows which data is for the jail and which
one is for the loopback.
Services (such as apache) listen on certain ports for
connections. For example, the default port for the HTTP
protocol is 80. So, when someone is trying to open a
connection to your IP address on port 80, your kernel
looks it up in its table of listening TCP sockets and
find the apache process which is running inside the jail.
So the connection is handed to the jail.
(This is a bit oversimplifying, but basically that's how
it works.)
OK. This is clear. And it explains how multiple jails can share the same
address.
Post by Oliver Fromme
Post by Brice ERRANDONEA
Despite the sshd_enable="YES" line, I can't ssh from the host to the
jail. Well, I can... The first time I did it, I was asked if I wanted
to add the jail to the list of known hosts. I did it. No problem
there. But, immediatly after that, instead of displaying "login :",
the system displayed "passwd :".
That's normal. ssh never asks for the login. You can use the -l
option if you need to specify a different user name (or put it in your
~/.ssh/config).
Of course. I'm loosing my mind with all that jail trouble. It works perfectly
well with le -l option.
Post by Oliver Fromme
Some paranoid people have a special "login jail". They
ssh into the login jail, then log into the host or into
other jails from there. The host accepts ssh only from
localhost. But please forget this immediately; we don't
want to make things more complicated than necessary.
I thought it was intended to be impossible to access the host from the jail. But
you're right : I'll forget that.

So, we're progressing. But the problem is not over yet. Any other idea ?

Have a good evening, anyway.

Brice
Oliver Fromme
2010-08-11 20:55:11 UTC
Permalink
Post by Brice ERRANDONEA
Post by Valentin Bud
sysctl security.jail.allow_raw_sockets=1
I did it but ping still doesn't work.
Which IP address are you using for the jail now?

If you're using 127.0.0.1, you can only ping the host's
own IP addresses, because packets with a localnet IP
never leave a machine.

If you're using the "real" address (192.168.1.38) for
the jail, then you should be able to ping all addresses
that you can ping from the host. I just did a quick
test on my machine; it has the IP address 172.20.0.2
(which is being translated with NAT on my router, but
that doesn't matter):

HOST# sysctl security.jail.allow_raw_sockets=1
security.jail.allow_raw_sockets: 0 -> 1
HOST# jail / testjail 172.20.0.2 /bin/sh -E
# ping www.google.com
PING www.l.google.com (66.102.13.105): 56 data bytes
64 bytes from 66.102.13.105: icmp_seq=0 ttl=54 time=31.196 ms
64 bytes from 66.102.13.105: icmp_seq=1 ttl=54 time=25.553 ms
64 bytes from 66.102.13.105: icmp_seq=2 ttl=54 time=27.086 ms
Post by Brice ERRANDONEA
Post by Valentin Bud
Post by Brice ERRANDONEA
192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail.
Well, localnet addresses are not routed. If you give your
jail a localnet address, it won't be able to access the
network outside of the host. (Unless you take measures
to rewrite/translate the addresses and forward them.)
That's why DNS and portsnap don't work.
I suggest using the address 192.168.1.38 for the jail,
at least during installation. Make sure that the file
/etc/resolv.conf inside the jail is correct, so DNS will
work. Copying it from the host should be sufficient.
Isn't 192.168.1.38 a localnet address too ?
It's a private address (RFC 1918). I assume that you've got
a NAT router that translates it to a public IP address.
Post by Brice ERRANDONEA
Do you mean I should use the public ip of my computer here ?
Do you have one? So far you only mentioned 192.168.1.38.
Post by Brice ERRANDONEA
I thought it was intended to be impossible to access the host from the jail.
It depends on what you want to do with the jail. Jails can
be used for vastly different purposes.
Post by Brice ERRANDONEA
But you're right : I'll forget that.
Good. :-)

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"Clear perl code is better than unclear awk code; but NOTHING
comes close to unclear perl code" (taken from comp.lang.awk FAQ)
Brice ERRANDONEA
2010-08-12 08:03:11 UTC
Permalink
192.168.1.38 is the private address of rl0 on my host. 93.0.168.242 is the
public one. I tried both as the jail's address. With the private one, neither
portsnap nor ping work at all.

With the public one, I get this result :


FreeBSD# sysctl security.jail.allow_raw_sockets=1
security.jail.allow_raw_sockets: 0 -> 1
FreeBSD# /etc/rc.d/jail onestart server
Configuring jails:.
Starting jails: MaPrison.
FreeBSD# jexec 1 portsnap fetch
jexec: jail_attach(1): Invalid argument
FreeBSD# jls
JID IP Address Hostname Path
2 93.0.168.242 MaPrison /usr/prison
FreeBSD# jexec 2 portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... none found.
Fetching public key from portsnap.FreeBSD.org... failed.
No mirrors remaining, giving up.
FreeBSD# jexec 2 ping www.yahoo.fr
ping: cannot resolve www.yahoo.fr: Host name lookup failure
FreeBSD# jexec 2 ping 69.147.83.33
PING 69.147.83.33 (69.147.83.33): 56 data bytes

Then, nothing during a few minutes, so I used :

^C
--- 69.147.83.33 ping statistics ---
32 packets transmitted, 0 packets received, 100.0% packet loss

Data can be sent to the net now but it seems they can't come back.

I also tried after opening the jail the same way you do :

FreeBSD# jail /usr/prison MaPrison 93.0.168.242 /bin/sh -E
# ping 69.147.83.33
PING 69.147.83.33 (69.147.83.33): 56 data bytes
^C
--- 69.147.83.33 ping statistics ---
30 packets transmitted, 0 packets received, 100.0% packet loss
# portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... none found.
Fetching public key from portsnap.FreeBSD.org... failed.
No mirrors remaining, giving up.
#




________________________________
De : Oliver Fromme <***@lurza.secnetix.de>
À : freebsd-***@FreeBSD.ORG; ***@yahoo.fr
Envoyé le : Mer 11 août 2010, 22h 55min 11s
Objet : Re: How to connect a jail to the web ?
Post by Brice ERRANDONEA
Post by Valentin Bud
sysctl security.jail.allow_raw_sockets=1
I did it but ping still doesn't work.
Which IP address are you using for the jail now?

If you're using 127.0.0.1, you can only ping the host's
own IP addresses, because packets with a localnet IP
never leave a machine.

If you're using the "real" address (192.168.1.38) for
the jail, then you should be able to ping all addresses
that you can ping from the host. I just did a quick
test on my machine; it has the IP address 172.20.0.2
(which is being translated with NAT on my router, but
that doesn't matter):

HOST# sysctl security.jail.allow_raw_sockets=1
security.jail.allow_raw_sockets: 0 -> 1
HOST# jail / testjail 172.20.0.2 /bin/sh -E
# ping www.google.com
PING www.l.google.com (66.102.13.105): 56 data bytes
64 bytes from 66.102.13.105: icmp_seq=0 ttl=54 time=31.196 ms
64 bytes from 66.102.13.105: icmp_seq=1 ttl=54 time=25.553 ms
64 bytes from 66.102.13.105: icmp_seq=2 ttl=54 time=27.086 ms
Post by Brice ERRANDONEA
Post by Valentin Bud
Post by Brice ERRANDONEA
192.168.1.38 is the host's ip so I use 127.0.0.1 for the jail.
Well, localnet addresses are not routed. If you give your
jail a localnet address, it won't be able to access the
network outside of the host. (Unless you take measures
to rewrite/translate the addresses and forward them.)
That's why DNS and portsnap don't work.
I suggest using the address 192.168.1.38 for the jail,
at least during installation. Make sure that the file
/etc/resolv.conf inside the jail is correct, so DNS will
work. Copying it from the host should be sufficient.
Isn't 192.168.1.38 a localnet address too ?
It's a private address (RFC 1918). I assume that you've got
a NAT router that translates it to a public IP address.
Post by Brice ERRANDONEA
Do you mean I should use the public ip of my computer here ?
Do you have one? So far you only mentioned 192.168.1.38.
Post by Brice ERRANDONEA
I thought it was intended to be impossible to access the host from the jail.
It depends on what you want to do with the jail. Jails can
be used for vastly different purposes.
Post by Brice ERRANDONEA
But you're right : I'll forget that.
Good. :-)

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"Clear perl code is better than unclear awk code; but NOTHING
comes close to unclear perl code" (taken from comp.lang.awk FAQ)
_______________________________________________
freebsd-***@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-***@freebsd.org"
Oliver Fromme
2010-08-12 12:52:00 UTC
Permalink
Post by Brice ERRANDONEA
192.168.1.38 is the private address of rl0 on my host. 93.0.168.242 is the
public one. I tried both as the jail's address. With the private one, neither
portsnap nor ping work at all.
[...]
FreeBSD# jexec 2 ping www.yahoo.fr
ping: cannot resolve www.yahoo.fr: Host name lookup failure
FreeBSD# jexec 2 ping 69.147.83.33
PING 69.147.83.33 (69.147.83.33): 56 data bytes
[...]
32 packets transmitted, 0 packets received, 100.0% packet loss
Please show the _complete_ output from "ifconfig" and "netstat -rnfinet".

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

PI:
int f[9814],b,c=9814,g,i;long a=1e4,d,e,h;
main(){for(;b=c,c-=14;i=printf("%04d",e+d/a),e=d%a)
while(g=--b*2)d=h*b+a*(i?f[b]:a/5),h=d/--g,f[b]=d%g;}
Brice ERRANDONEA
2010-08-12 14:35:37 UTC
Permalink
Here they are.

On the host, when the jail is not running :

%ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fwe0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:11:06:99:8a:ff
ch 1 dma -1
fwip0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lladdr 0.11.6.66.0.99.8a.ff.a.2.ff.fe.0.0.0.0
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
%netstat -rnfinet
Routing tables

Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.1.1 UGS 16 434 rl0
127.0.0.1 link#5 UH 0 20 lo0
192.168.1.0/24 link#1 U 1 98 rl0
192.168.1.38 link#1 UHS 0 0 lo0

On the host when the jail is running :

FreeBSD# jls
JID IP Address Hostname Path
1 93.0.168.242 MaPrison /usr/prison
FreeBSD# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
inet 93.0.168.242 netmask 0xffffffff broadcast 93.0.168.242
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fwe0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:11:06:99:8a:ff
ch 1 dma -1
fwip0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lladdr 0.11.6.66.0.99.8a.ff.a.2.ff.fe.0.0.0.0
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
FreeBSD# netstat -rnfinet
Routing tables

Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.1.1 UGS 0 474 rl0
93.0.168.242 link#1 UHS 0 20 lo0 =>
93.0.168.242/32 link#1 U 0 0 rl0
127.0.0.1 link#5 UH 0 20 lo0
192.168.1.0/24 link#1 U 0 102 rl0
192.168.1.38 link#1 UHS 0 0 lo0

In the jail (running, of course) :

FreeBSD# jexec 1 ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 93.0.168.242 netmask 0xffffffff broadcast 93.0.168.242
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fwe0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:11:06:99:8a:ff
ch 1 dma -1
fwip0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
lladdr 0.11.6.66.0.99.8a.ff.a.2.ff.fe.0.0.0.0
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
FreeBSD# jexec 1 netstat -rnfinet
Routing tables

Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.1.1 UGS 0 480 rl0
93.0.168.242 link#1 UHS 0 20 lo0 =>
93.0.168.242/32 link#1 U 0 0 rl0
127.0.0.1 link#5 UH 0 20 lo0
192.168.1.0/24 link#1 U 0 102 rl0
192.168.1.38 link#1 UHS 0 0 lo0

Do you find what's wrong ?

Brice





________________________________
De : Oliver Fromme <***@lurza.secnetix.de>
À : freebsd-***@FreeBSD.ORG; ***@yahoo.fr
Envoyé le : Jeu 12 août 2010, 14h 52min 00s
Objet : Re: Re : How to connect a jail to the web ?
Post by Brice ERRANDONEA
192.168.1.38 is the private address of rl0 on my host. 93.0.168.242 is the
public one. I tried both as the jail's address. With the private one, neither
portsnap nor ping work at all.
[...]
FreeBSD# jexec 2 ping www.yahoo.fr
ping: cannot resolve www.yahoo.fr: Host name lookup failure
FreeBSD# jexec 2 ping 69.147.83.33
PING 69.147.83.33 (69.147.83.33): 56 data bytes
[...]
32 packets transmitted, 0 packets received, 100.0% packet loss
Please show the _complete_ output from "ifconfig" and "netstat -rnfinet".

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

PI:
int f[9814],b,c=9814,g,i;long a=1e4,d,e,h;
main(){for(;b=c,c-=14;i=printf("%04d",e+d/a),e=d%a)
while(g=--b*2)d=h*b+a*(i?f[b]:a/5),h=d/--g,f[b]=d%g;}
_______________________________________________
freebsd-***@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-***@freebsd.org"
Oliver Fromme
2010-08-12 15:52:24 UTC
Permalink
Post by Brice ERRANDONEA
%ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (100baseTX <full-duplex>)
OK, so 192.168.1.38 is the only (non-localnet) IP address that
you have. You should use that one for your jail.
Post by Brice ERRANDONEA
FreeBSD# jls
JID IP Address Hostname Path
1 93.0.168.242 MaPrison /usr/prison
FreeBSD# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
inet 93.0.168.242 netmask 0xffffffff broadcast 93.0.168.242
media: Ethernet autoselect (100baseTX <full-duplex>)
Where did you get that second IP address from? Did you just
add it manually? Or is that the address that your gateway
(DSL router, whatever) got assigned from your ISP?

I assume that IP address is not really routed to your host,
but that NAT (Network Address Translation) is used on your
router. So you cannot use that address on the host.
(If that's not true, please exlain the structure of your
network in more detail.)

So, if my assumptions are true, you must use the address
192.168.1.38 for your jail. Make sure that DNS is working
inside the jail ... It should be sufficient to copy
/etc/resolv.conf from the host to /usr/prison/etc/resolv.conf

If it still doesn't work: Are you using any packet filter
(ipfw, ipf, pf)? If so, please show the complete list of
rules.

Otherwise, it might help to run tcpdump(1) on the host, so
you can see the actual packets that are transmitted and
received.

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"C++ is the only current language making COBOL look good."
-- Bertrand Meyer
Brice ERRANDONEA
2010-08-12 16:48:51 UTC
Permalink
Post by Oliver Fromme
Where did you get that second IP address from? Did you just
add it manually? Or is that the address that your gateway
(DSL router, whatever) got assigned from your ISP?
I added it manually in rc.conf (on the host) :

hostname="FreeBSD.ici"
ifconfig_rl0="DHCP"
keymap="fr.iso.acc" (yes, I'm french)
moused_enable="YES"
saver="dragon"
hald_enable="YES"
dbus_enable="YES"
devfs_system_ruleset="localrules"

jail_enable="NO"
jail_list="MaPrison"
jail_interface="rl0"
jail_devfs_ruleset="devfsrules_jail"
jail_devfs_enable="YES"

jail_server_rootdir="/usr/prison"
jail_server_hostname="MaPrison"
jail_server_ip="93.0.168.242"

I choosed it because that's my computer's public ip, at least according to this
website : http://whatismyipaddress.com/
Post by Oliver Fromme
I assume that IP address is not really routed to your host,
but that NAT (Network Address Translation) is used on your
router. So you cannot use that address on the host.
(If that's not true, please exlain the structure of your
network in more detail.)
My "network" is VERY simple. I've got a modem (or "box") provided by my phone
company. It's called a "neufbox" and acts as a gateway. The computer with
FreeBSD is connected to this "box" through an ethernet cable. Two other
computers are connected to it via wifi.
Post by Oliver Fromme
So, if my assumptions are true, you must use the address
192.168.1.38 for your jail. Make sure that DNS is working
inside the jail ... It should be sufficient to copy
/etc/resolv.conf from the host to /usr/prison/etc/resolv.conf
OK, I'll try this.
Post by Oliver Fromme
If it still doesn't work: Are you using any packet filter
(ipfw, ipf, pf)? If so, please show the complete list of
rules.
No, I don't. I've tried pf but you told it was not necessary.
Post by Oliver Fromme
Otherwise, it might help to run tcpdump(1) on the host, so
you can see the actual packets that are transmitted and
received.
Allright. I try it too.

Good bye for the moment and thanks for your help.

Brice
Brice ERRANDONEA
2010-08-14 10:02:29 UTC
Permalink
I had a break with this yesterday. I've just tried your suggestions. It still
doesn't work but the error message has changed.
Post by Oliver Fromme
Post by Brice ERRANDONEA
FreeBSD# jls
JID IP Address Hostname Path
1 93.0.168.242 MaPrison /usr/prison
FreeBSD# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
inet 93.0.168.242 netmask 0xffffffff broadcast 93.0.168.242
media: Ethernet autoselect (100baseTX <full-duplex>)
Where did you get that second IP address from? Did you just
add it manually? Or is that the address that your gateway
(DSL router, whatever) got assigned from your ISP?
I added it manually in rc.conf (on the host) :

jail_server_rootdir="/usr/prison"
jail_server_hostname="MaPrison"
jail_server_ip="93.0.168.242"

I choosed it because that's my computer's public ip, at least according to this
website : http://whatismyipaddress.com/
Post by Oliver Fromme
I assume that IP address is not really routed to your host,
but that NAT (Network Address Translation) is used on your
router. So you cannot use that address on the host.
(If that's not true, please exlain the structure of your
network in more detail.)
My network is very simple. I've got a kind of modem provided by my phone
company. It's called a "neufbox" and acts as a gateway. Its address is
192.168.1.1. This "neufbox" is connected to :

- the phone network
- a phone
- the FreeBSD computer through an ethernet wire
- two other computers via wifi

When I browse address 192.168.1.1 with firefox, I can see a page telling this
the neufbox, that internet and the phone are working, that the tv is not
connected (that's true) and that it's public ip address is 93.0.168.242. It also
gives its MAC address and various other infos.
Post by Oliver Fromme
So, if my assumptions are true, you must use the address
192.168.1.38 for your jail.
OK. In /etc/rc.conf, I changed this line (see above) :
jail_server_ip="198.168.1.38"
Post by Oliver Fromme
Make sure that DNS is working
inside the jail ... It should be sufficient to copy
/etc/resolv.conf from the host to /usr/prison/etc/resolv.conf
/etc/resolv.conf only contains this single line : nameserver 192.168.1.1

I placed a copy of this file in the jail.

After these changes and a complete reboot, I launched the jail and tried a
portsnap fetch :

FreeBSD# /etc/rc.d/jail onestart server
Configuring jails:.
Starting jails: MaPrison.
FreeBSD# jls
JID IP Address Hostname Path
1 192.168.1.38 MaPrison /usr/prison
FreeBSD# jexec 1 portsnap fetch
Looking up portsnap.FreeBSD.org mirrors...
/usr/src/lib/bind/isc/../../../contrib/bind9/lib/isc/unix/socket.c:1699:
internal_send: 192.168.1.1#53: Invalid argument

/usr/src/lib/bind/isc/../../../contrib/bind9/lib/isc/unix/socket.c:1699:
internal_send: 192.168.1.1#53: Invalid
argument

none
found.

Fetching public key from portsnap.FreeBSD.org...
failed.

No mirrors remaining, giving
up.

FreeBSD#

Then, firefox (on the host) was no longer able to browse. I tried this on the
host :

FreeBSD# ping www.freebsd.org
ping: cannot resolve www.freebsd.org: Host name lookup failure

In other words, it appeared that DNS was no longer working, even on the host.

I rebooted again. This time, I didn't launch the jail. ping and Firefox worked
perfectly well on the host as they had always did before.
Post by Oliver Fromme
If it still doesn't work: Are you using any packet filter
(ipfw, ipf, pf)? If so, please show the complete list of
rules.
No, I don't. You told me it was not necessary.
Post by Oliver Fromme
Otherwise, it might help to run tcpdump(1) on the host, so
you can see the actual packets that are transmitted and
received.
Here's what tcpdump says when the jail is NOT running (but Firefox is) :

FreeBSD# tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on rl0, link-type EN10MB (Ethernet), capture size 96 bytes
09:08:50.300910 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 263
09:08:50.301378 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 335
09:08:50.301822 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 331
09:08:50.302275 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 311
09:08:50.302933 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 343
09:08:50.303485 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 325
09:08:50.303938 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 327
09:08:50.304383 IP neufbox.32774 > 239.255.255.250.1900: UDP, length 327
09:08:50.858573 IP FreeBSD.22077 > neufbox.domain: 24445+ PTR?
250.255.255.239.in-addr.arpa. (46)
09:08:50.906882 IP neufbox.domain > FreeBSD.22077: 24445 NXDomain 0/1/0 (103)
09:08:50.917164 IP FreeBSD.59750 > neufbox.domain: 24446+ PTR?
1.1.168.192.in-addr.arpa. (42)
09:08:50.918253 IP neufbox.domain > FreeBSD.59750: 24446* 1/0/0 PTR[|domain]
09:08:51.917971 IP FreeBSD.32837 > neufbox.domain: 24447+ PTR?
38.1.168.192.in-addr.arpa. (43)
09:08:51.918870 IP neufbox.domain > FreeBSD.32837: 24447* 1/0/0 (64)
^C
14 packets captured
14 packets received by filter
0 packets dropped by kernel
FreeBSD#

Then, I started the jail. Firefox immediatly stopped being able to browse
websites. I tried a tcpdump on the host while running portsnap fetch in the jail
:

FreeBSD# tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on rl0, link-type EN10MB (Ethernet), capture size 96 bytes
09:43:50.333169 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 263
09:43:50.333621 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 335
09:43:50.334064 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 331
09:43:50.334499 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 311
09:43:50.334966 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 343
09:43:50.335402 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 325
09:43:50.335944 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 327
09:43:50.336560 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 327
09:44:20.333341 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 263
09:44:20.333807 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 335
09:44:20.334246 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 331
09:44:20.334684 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 311
09:44:20.335165 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 343
09:44:20.335603 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 325
09:44:20.336040 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 327
09:44:20.336480 IP 192.168.1.1.32774 > 239.255.255.250.1900: UDP, length 327
^C
16 packets captured
16 packets received by filter
0 packets dropped by kernel
FreeBSD#

If you compare these two tcpdump, you can see that the word "neufbox" is
replaced by 192.168.1.1. It confirms that DNS is no longer running.

Not easy...

Brice



________________________________
De : Oliver Fromme <***@lurza.secnetix.de>
À : freebsd-***@FreeBSD.ORG; ***@yahoo.fr
Envoyé le : Jeu 12 août 2010, 17h 52min 24s
Objet : Re: Re : Re : How to connect a jail to the web ?
Post by Oliver Fromme
%ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (100baseTX <full-duplex>)
OK, so 192.168.1.38 is the only (non-localnet) IP address that
you have. You should use that one for your jail.
Post by Oliver Fromme
FreeBSD# jls
JID IP Address Hostname Path
1 93.0.168.242 MaPrison /usr/prison
FreeBSD# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 00:11:09:15:72:6a
inet 192.168.1.38 netmask 0xffffff00 broadcast 192.168.1.255
inet 93.0.168.242 netmask 0xffffffff broadcast 93.0.168.242
media: Ethernet autoselect (100baseTX <full-duplex>)
Where did you get that second IP address from? Did you just
add it manually? Or is that the address that your gateway
(DSL router, whatever) got assigned from your ISP?

I assume that IP address is not really routed to your host,
but that NAT (Network Address Translation) is used on your
router. So you cannot use that address on the host.
(If that's not true, please exlain the structure of your
network in more detail.)

So, if my assumptions are true, you must use the address
192.168.1.38 for your jail. Make sure that DNS is working
inside the jail ... It should be sufficient to copy
/etc/resolv.conf from the host to /usr/prison/etc/resolv.conf

If it still doesn't work: Are you using any packet filter
(ipfw, ipf, pf)? If so, please show the complete list of
rules.

Otherwise, it might help to run tcpdump(1) on the host, so
you can see the actual packets that are transmitted and
received.

Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

"C++ is the only current language making COBOL look good."
-- Bertrand Meyer
_______________________________________________
freebsd-***@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-***@freebsd.org"
Continue reading on narkive:
Loading...