SSL ca-certificates missing but adding has unpredictable results


Recommended Posts

I am primarily working with V6 beta however the same was true with v5.X. unRAID does not ship with the ca-certificates bundle. It should and will once we work out why adding it doesn't work as expected.

 

Installing "ca-certificates-20130906-noarch-1.txz" populates /etc/ssl as expected however even after it is added all the big player sites I have tested still fail the SSL 3 way handshake.

 

Something is wrong and at this point I cannot work out what.

 

Initially I thought that the bundle was out of date and I had some success manually installing certs but checking it against Debian the dates matches the unstable bleeding edge branch "20130906" suggesting it is not a deprecation problem.

 

More interestingly Debian stable works which is an even older cert bundle

 

You can test this by trying:

 

wget https://raw.github.com/

 

And you will get a warning:

 

ERROR: cannot verify raw.github.com's certificate, issued by ‘/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance CA-3’:
  Unable to locally verify the issuer's authority.

 

Obviously you can add "--no-check-certificate" but specifically that "almost" breaks the whole point of SSL and we should fix this the proper way.

 

Ideas? Any SSL CA experts?

 

Update: Looks like results might be different with wget and curl.

Link to comment

Edit: I should have read more thoroughly. I see now you already tried installing the certs. Try running c_rehash.

 

You need to update your root certificates (this is normal). It looks like the latest one is from Sept 2013. Slackware tends to lag behind a bit.

 

Install the latest with trolley:

 

trolley info ca-certificates ">0"

 

Or manually:

 

wget -P /boot/extra http://slackware.cs.utah.edu/pub/slackware/slackware64-14.1/slackware64/n/ca-certificates-20130906-noarch-1.txz
installpkg /boot/extra/ca-certificates-20130906-noarch-1.txz

 

Sometimes you need to go the extra mile and relink the certificates. Run c_rehash (an openssl utility. Requires perl).

 

c_rehash

Link to comment

In fact the install script on ca-certificates package should do it... but you need perl package (and openssl for unraid 5.x) installed before installing ca-certificates package, I did found it trying to get it sorted on 5.x some months ago but should be similar on 6.x.

 

install script will actually run:

/usr/sbin/update-ca-certificates --fresh 1> /dev/null 2> /dev/null

 

and that 'update-ca-certificates' script will use c_rehash perl script as nicinabox did mention.

 

p.s. due to these sort of things think could be nice if packages on extra folder get loaded sorted by filename... then we would be able to add a prefix to force some packages loading on specific order.

Link to comment

nicinabox and nars thanks for taking the time to help :)

 

So essentially it is nailed. Long story short update-ca-certificates will silently fail if perl isnt there. One of the times it worked I had Perl temp installed for something else which explains the odd results I was seeing.

 

I am not sure where to go with this. We need ca-certificates in V5 and V6 but we also dont want quite so many of these dependencies.

 

I have pointed Tom at this thread an hopefully he can incorporate the update into his build then remove the deps before the final packaging.

Link to comment

Right, need the ca-certificates package (which installs a bunch of already trusted certificates), and perl so that 'update-ca-certificates' will run.  Also I guess we need a way to dynamically add certificates, right?  For example, we could use a webGui plugin that will let you add certificates (stored on the flash somewhere) and run update-ca-certificates when a new is defined.

Link to comment

p.s. due to these sort of things think could be nice if packages on extra folder get loaded sorted by filename... then we would be able to add a prefix to force some packages loading on specific order.

 

This is a really challenging problem. It's probably not a good practice to munge the file names (vendor files shouldn't be mucked with). If we went down that road, it would require everyone to rename their packages the same way. It makes it really difficult to detect what's actually installed (/var/log/packages isn't necessarily accurate after pruning is done for the build), which makes installing dependencies accurately even harder.

 

In an ideal world, packages would just install their dependencies first. Since Slackware doesn't natively support this, it puts us in a really awkward situation.

 

Some order is probably better than no order, but I'd be hesitant to rely on alphabetical as a fix.

Link to comment

p.s. due to these sort of things think could be nice if packages on extra folder get loaded sorted by filename... then we would be able to add a prefix to force some packages loading on specific order.

 

This is a really challenging problem. It's probably not a good practice to munge the file names (vendor files shouldn't be mucked with). If we went down that road, it would require everyone to rename their packages the same way. It makes it really difficult to detect what's actually installed (/var/log/packages isn't necessarily accurate after pruning is done for the build), which makes installing dependencies accurately even harder.

 

In an ideal world, packages would just install their dependencies first. Since Slackware doesn't natively support this, it puts us in a really awkward situation.

 

Some order is probably better that no order, but I'd be hesitant to rely on alphabetical as a fix.

The way slack solves this is by arranging their packages in subdirectories:

http://slackware.cs.utah.edu/pub/slackware/slackware64-current/slackware64/

 

In this specific case, the ca-certificates package is in n/ and the perl package is in d/ so perl gets installed before ca-certificates.  Yes this is pretty unsophisticated but dirt simple and works ok at least for installing the base OS.

 

 

Link to comment

In an ideal world, packages would just install their dependencies first. Since Slackware doesn't natively support this, it puts us in a really awkward situation.

 

This is probably the wrong thread for this, but let me address this issue.  There is another wrinkle having to do with package management.  The wrinkle is that packages (in slack) are built from source.  Of course you can download individual pre-built packages that you just install.  But you can also download the source and the files used to create the package:

http://slackware.cs.utah.edu/pub/slackware/slackware64-current/source/

 

The build script for virtually all packages use 'configure' to set compile-time features.  It is very easy to customize this.  For example, I build the samba package from source for unRaid because I need to tweak the config to include support for "idhash" method of translating windows SID to linux UID/GID.

 

The point is that the set of underlying dependencies can change depending on how you built the package.

 

My thinking is this: it's the developer of a package that already knows (or should know imho) what dependencies exist for their package.  Since they already know, well they just name those packages in the plugin file.  When the plugin gets installed, it checks if the dependent package already exists and if so, doesn't install it (this is done by 'upgradepkg').

 

My other thought is, suppose you develop "whiz-bang" plugin, and during development you discover whiz-bang requires 400MB of dependencies.  Maybe you need to re-think whiz-bang and whether it's appropriate to install on a possibly-memory-limited "NAS Appliance".  In other words, I want plugin developers to have to think about what they are causing to be installed.

Link to comment

p.s. due to these sort of things think could be nice if packages on extra folder get loaded sorted by filename... then we would be able to add a prefix to force some packages loading on specific order.

 

This is a really challenging problem. It's probably not a good practice to munge the file names (vendor files shouldn't be mucked with). If we went down that road, it would require everyone to rename their packages the same way. It makes it really difficult to detect what's actually installed (/var/log/packages isn't necessarily accurate after pruning is done for the build), which makes installing dependencies accurately even harder.

 

In an ideal world, packages would just install their dependencies first. Since Slackware doesn't natively support this, it puts us in a really awkward situation.

 

Some order is probably better that no order, but I'd be hesitant to rely on alphabetical as a fix.

The way slack solves this is by arranging their packages in subdirectories:

http://slackware.cs.utah.edu/pub/slackware/slackware64-current/slackware64/

 

In this specific case, the ca-certificates package is in n/ and the perl package is in d/ so perl gets installed before ca-certificates.  Yes this is pretty unsophisticated but dirt simple and works ok at least for installing the base OS.

 

Yes, I understand nicinabox 'fear' on messing with package filenames, in some cases may indeed get out of control with different renamed files added from different plugins, etc... Then what about, loading packages from 'extra' folder still sorted by name, and... allowing to load them from subdirectories? allowing us to just replicate similar directory structure (a, d, l, n, ...) also in 'extra' folder, without messing with package filenames?  Sure there will always still exist the risk of duplicates, inside and outside 'proper' directory structure, but probably a lot less problem and if 'upgradepkg --install-new' used to install them then would end being no issue as second one would just be ignored? Just an idea though.

Link to comment

p.s. due to these sort of things think could be nice if packages on extra folder get loaded sorted by filename... then we would be able to add a prefix to force some packages loading on specific order.

 

This is a really challenging problem. It's probably not a good practice to munge the file names (vendor files shouldn't be mucked with). If we went down that road, it would require everyone to rename their packages the same way. It makes it really difficult to detect what's actually installed (/var/log/packages isn't necessarily accurate after pruning is done for the build), which makes installing dependencies accurately even harder.

 

In an ideal world, packages would just install their dependencies first. Since Slackware doesn't natively support this, it puts us in a really awkward situation.

 

Some order is probably better that no order, but I'd be hesitant to rely on alphabetical as a fix.

The way slack solves this is by arranging their packages in subdirectories:

http://slackware.cs.utah.edu/pub/slackware/slackware64-current/slackware64/

 

In this specific case, the ca-certificates package is in n/ and the perl package is in d/ so perl gets installed before ca-certificates.  Yes this is pretty unsophisticated but dirt simple and works ok at least for installing the base OS.

 

Wait, I'm not sure that works across the board. Ruby is in d/ and depends on libyaml in l/, so that wouldn't work by going alphabetical.

 

The release readme lays it out:

 

|-- slackware64/         This directory contains the core software packages
|   |                    for Slackware 14.1.
|   |
|   |-- a/               The A (base) package series.
|   |-- ap/              The AP (applications) package series.
|   |-- d/               The D (development) package series.
|   |-- e/               The E (GNU Emacs) package series.
|   |-- f/               The F (FAQ/Documentation) package series.
|   |-- k/               The K (kernel source) package series.
|   |-- kde/             The KDE package series.
|   |-- kdei/            The KDE internationalization package series.
|   |-- l/               The L (libraries) package series.
|   |-- n/               The N (networking) package series.
|   |-- t/               The T (TeX) package series.
|   |-- tcl/             The TCL (Tcl/Tk and related) package series.
|   |-- x/               The X (X Window System) package series.
|   |-- xap/             The XAP (X applications) package series.
|   |-- xfce/            The XFCE (Xfce desktop) package series.
|   `-- y/               The Y (BSD games) package series.

 

I'm really interested in discussing this further. Should we branch this topic?

Link to comment
  • 2 weeks later...
  • 2 months later...

Long overdue update.

 

unRAID 6 beta 4 almost works. Everything is there but  you need to run "update-ca-certificates" on each boot for it to take.

 

Tom can you look into this as its all that stands in the way of us telling people to stop using "wget --no-check-certificate"

Link to comment

Long overdue update.

 

unRAID 6 beta 4 almost works. Everything is there but  you need to run "update-ca-certificates" on each boot for it to take.

 

Tom can you look into this as its all that stands in the way of us telling people to stop using "wget --no-check-certificate"

If I run update-ca-certificates during release build process, so that /etc/ssl/certs is populated, does that preclude need to run it at boot time (wondering if update-ca-certificates access some server somewhere during it's execution)?

Link to comment