unix - Unable to build a working FIPS capable OpenSSL on HP-UX

Question

I am building openssl-1.0.2f with openssl-fips-2.0.12 (I am going to talk about this configuration in the following lines, but at the end of the post I'll specify all the configurations that I tried), on HP-UX11.31 (pa-risc2 ([HPE]: pa-risc1.1 pa-risc2.0)). Everything is good, but when I try using it (in FIPS mode), it doesn't work.

Note: Given the fact that cwd is set to the build folder (not the installation folder where RPATH points to), I need to instruct the linker where to search for libs (SHLIB_PATH):

[%__OPENSSL_MACHINE_PROMPT%]> OPENSSL_FIPS=1 SHLIB_PATH=./lib ./bin/openssl version -a
2063867464:error:2D06B071:FIPS routines:FIPS_check_incore_fingerprint:fingerprint does not match segment aliasing:fips.c:224:

Note: Instead of displaying any path, I'm replacing it by a meaningful placeholder (name starting with __OPENSSL) surrounded by % signs (the equivalent of Win env vars - don't want to create confusion if any actual Ux env vars might be involved).

Here's the output of the "same" command without setting FIPS (OPENSSL_FIPS=1) mode:

[%__OPENSSL_MACHINE_PROMPT%]> SHLIB_PATH=./lib ./bin/openssl version -a
OpenSSL 1.0.2f-fips  28 Jan 2016
  built on: Fri Feb 26 09:53:34 2016
  platform: hpux-parisc2-gcc
  options:  bn(64,64) rc4(ptr,char) des(ptr,risc1,16,long) blowfish(idx)
  compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS  -DDSO_DL -fPIC -D_REENTRANT -march=2.0 -O3 -DB_ENDIAN -D_REENTRANT -I%__OPENSSL_BUILD_PATH%/include
  OPENSSLDIR: "%__OPENSSL_PREFIX_DIR%"

This occurs on all the machines I've tried running it (including the very machine I've built it on):

[%__OPENSSL_BUILD_MACHINE_PROMPT%]> uname -a
HP-UX hpux1131 B.11.31 U 9000/800 629887774 unlimited-user license

gcc version (native linker (ld_pa) used):

[%__OPENSSL_BUILD_MACHINE_PROMPT%]> gcc -v
  Using built-in specs.
  Target: hppa2.0w-hp-hpux11.31
  Configured with: ../gcc-4.2.4/configure --disable-shared --with-gnu-as --with-as=%__GCC_PREFIX_PATH%/bin/as --with-ld=/bin/ld --disable-nls --enable-threads=posix --prefix=%__GCC_PREFIX_PATH% --with-local-prefix=%__GCC_PREFIX_PATH%
  Thread model: posix
  gcc version 4.2.4`

• Here's the openssl-fips-2.1.12 configurator's output:

  ./config no-asm
  Operating system: 9000/800-hp-hpux1x
    Auto Configuring fipsonly
    Auto Configuring fipsonly
    Configuring for hpux-parisc2-gcc
    Auto Configuring fipsonly
    Configuring for hpux-parisc2-gcc
        no-asm          [option]   OPENSSL_NO_ASM
        no-bf           [option]   OPENSSL_NO_BF (skip dir)
        no-camellia     [option]   OPENSSL_NO_CAMELLIA (skip dir)
        no-cast         [option]   OPENSSL_NO_CAST (skip dir)
        no-ec_nistp_64_gcc_128 [default]  OPENSSL_NO_EC_NISTP_64_GCC_128 (skip dir)
        no-gmp          [default]  OPENSSL_NO_GMP (skip dir)
        no-idea         [option]   OPENSSL_NO_IDEA (skip dir)
        no-jpake        [experimental] OPENSSL_NO_JPAKE (skip dir)
        no-krb5         [krb5-flavor not specified] OPENSSL_NO_KRB5
        no-md2          [option]   OPENSSL_NO_MD2 (skip dir)
        no-md5          [option]   OPENSSL_NO_MD5 (skip dir)
        no-mdc2         [option]   OPENSSL_NO_MDC2 (skip dir)
        no-rc2          [option]   OPENSSL_NO_RC2 (skip dir)
        no-rc4          [option]   OPENSSL_NO_RC4 (skip dir)
        no-rc5          [option]   OPENSSL_NO_RC5 (skip dir)
        no-rfc3779      [default]  OPENSSL_NO_RFC3779 (skip dir)
        no-ripemd       [option]   OPENSSL_NO_RIPEMD (skip dir)
        no-seed         [option]   OPENSSL_NO_SEED (skip dir)
        no-srp          [forced]   OPENSSL_NO_SRP (skip dir)
        no-ssl2         [forced]   OPENSSL_NO_SSL2 (skip dir)
        no-ssl3         [forced]   OPENSSL_NO_SSL3 (skip dir)
        no-store        [experimental] OPENSSL_NO_STORE (skip dir)
        no-tls1         [forced]   OPENSSL_NO_TLS1 (skip dir)
        no-tlsext       [forced]   OPENSSL_NO_TLSEXT (skip dir)
        no-zlib         [default]
        no-zlib-dynamic [default]
  

• And here's openssl-1.0.2f's:

  ./config fips shared --prefix=%__OPENSSL_PREFIX_DIR% no-rc5 no-mdc2 no-idea -fPIC no-asm --openssldir=%__OPENSSL_PREFIX_DIR%/openssl
  Operating system: 9000/800-hp-hpux1x
    Configuring for hpux-parisc2-gcc
    Configuring for hpux-parisc2-gcc
        no-asm          [option]   OPENSSL_NO_ASM
        no-ec_nistp_64_gcc_128 [default]  OPENSSL_NO_EC_NISTP_64_GCC_128 (skip dir)
        no-gmp          [default]  OPENSSL_NO_GMP (skip dir)
        no-idea         [option]   OPENSSL_NO_IDEA (skip dir)
        no-jpake        [experimental] OPENSSL_NO_JPAKE (skip dir)
        no-krb5         [krb5-flavor not specified] OPENSSL_NO_KRB5
        no-libunbound   [experimental] OPENSSL_NO_LIBUNBOUND (skip dir)
        no-md2          [default]  OPENSSL_NO_MD2 (skip dir)
        no-mdc2         [option]   OPENSSL_NO_MDC2 (skip dir)
        no-rc5          [option]   OPENSSL_NO_RC5 (skip dir)
        no-rfc3779      [default]  OPENSSL_NO_RFC3779 (skip dir)
        no-rsax         [forced]   OPENSSL_NO_RSAX (skip dir)
        no-sctp         [default]  OPENSSL_NO_SCTP (skip dir)
        no-ssl-trace    [default]  OPENSSL_NO_SSL_TRACE (skip dir)
        no-store        [experimental] OPENSSL_NO_STORE (skip dir)
        no-unit-test    [default]  OPENSSL_NO_UNIT_TEST (skip dir)
        no-zlib         [default]
        no-zlib-dynamic [default]
  

Important note: I've stated the problem that I have using openssl-1.0.2f + openssl-fips-2.0.12 on HP-UX11.31 om PA-RISC2. What else I've tried:

• openssl-1.0.1X (where X = [e..p]) + openssl-fips-2.0.5
• HP-UX11.31 or HP-UX11.11 on PA-RISC2
• no-asm configure flag specified/unspecified

Note: During debug, I've also modified fips_premain.c (and others), and (shocking :) ), the fingerprint produced by fips_premain_dso (compiled with -DFINGERPRINT_PREMAIN_DSO_LOAD) and the one computed at runtime don't match! I've also dumped the memory zone (in original or hex format) that the fingerprint is being computed on, and (of course) it differs (but so for I can't tell why).

Given the fact that it works (or it is supposed to work) - even if not being tested on pa-risc, but only on IA64 -, and extensive Google search didn't reveal anything truly relevant, I am 99.99 sure that it is related to the machine(s) in my environment.

However, can anyone give me some pointers?

@EDIT0: I mentioned that I reproduced the same problem on IA64; it was most likely a mistake. I recently (got a machine and) built it on HP-UX11.23 IA64 and it worked fine. The only problem is that cross architecture compatibility (build/run) is one way only: PA-RISC -> IA64.

Answer 1

Waitting for answers