Mailing List Archive

Author: sidney
Date: Tue Dec 6 11:05:53 2022
New Revision: 1905790

URL: http://svn.apache.org/viewvc?rev=1905790&view=rev
Log:
Bug 8056 - commit Github Actions that can be used in a fork of our Github mirror to run regression tests on Github action runners

Added:
spamassassin/trunk/.github/
spamassassin/trunk/.github/actions/
spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/
spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/action.yml
spamassassin/trunk/.github/actions/setup_macos_for_sa_test/
spamassassin/trunk/.github/actions/setup_macos_for_sa_test/action.yml
spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/
spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/action.yml
spamassassin/trunk/.github/workflows/
spamassassin/trunk/.github/workflows/makefile_one_native.yml
Modified:
spamassassin/trunk/MANIFEST.SKIP

Added: spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/action.yml
URL: http://svn.apache.org/viewvc/spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/action.yml?rev=1905790&view=auto
==============================================================================
--- spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/action.yml (added)
+++ spamassassin/trunk/.github/actions/install_cpan_dependencies_for_sa/action.yml Tue Dec 6 11:05:53 2022
@@ -0,0 +1,23 @@
+name: 'Install CPAN modules for SpamAssassin build and test Linux and Mac'
+description: 'Assuming OS and perl alreqady configured and installed, install from CPAN all modules needed for SapmAssassin build and test'
+runs:
+ using: 'composite'
+ steps:
+ - name: Install cpan modules
+ run: |
+ #cpanm App::cpanoutdated -nq --install-args="--install_path sbin=$HOME/.local/bin"
+ #hash -r
+ cpanm -n Module::Install Archive::Zip BSD::Resource BerkeleyDB Compress::Zlib DBI DB_File Devel::Cycle Digest::SHA Digest::SHA1 Email::Address::XS Encode::Detect Encode::Detect::Detector Geo::IP GeoIP2 GeoIP2::Database::Reader Geography::Countries HTML::Parser HTTP::Cookies HTTP::Daemon HTTP::Date HTTP::Negotiate IO::Socket::INET6 IO::Socket::SSL IO::String IP::Country IP::Country::DB_File LWP::Protocol::https LWP::UserAgent Mail::DKIM Mail::DMARC::PurePerl Math::Int128 MaxMind::DB::Reader::XS Net::CIDR::Lite Net::DNS Net::DNS::Nameserver Net::LibIDN Net::LibIDN2 Net::Patricia Net::Works::Network NetAddr::IP Params::Validate Sys::Hostname::Long Test::Perl::Critic Test::Pod Test::Pod::Coverage WWW::RobotRules Text::Diff Perl::Critic::Policy::Bangs::ProhibitBitwiseOperators Perl::Critic::Policy::Bangs::ProhibitDebuggingModules List::MoreUtils Perl::Critic::Policy::Perlsecret Perl::Critic::Policy::Compatibility::ProhibitThreeArgumentOpen Perl::Critic::Policy::Lax::ProhibitStr
ingyEval::ExceptForRequire Perl::Critic::Policy::ValuesAndExpressions::PreventSQLInjection Perl::Critic::Policy::ControlStructures::ProhibitReturnInDoBlock || echo "warn: errors installing cpan modules have been ignored"
+ cpanm Razor2::Client::Agent -n --install-args="DESTINSTALLSITESCRIPT=$HOME/.local/bin" || echo "warn: errors installing Razor2::Client::Agent have been ignored"
+ hash -r
+ razor-admin --create
+ razor-admin --register
+ cpanm Mail::SPF -n --install-args="--install_path sbin=$HOME/.local/bin" || echo "warn: errors installing Mail::SPF have been ignored"
+ cpanm --installdeps --notest . || echo "warn: errors installing required cpan modules have been ignored"
+ echo "Finished step"
+ shell: bash
+
+ - name: Show content of log files on Linux and Mac
+ if: ${{ failure() }}
+ run: cat $HOME/.cpanm/work/*/build.log
+ shell: bash

Added: spamassassin/trunk/.github/actions/setup_macos_for_sa_test/action.yml
URL: http://svn.apache.org/viewvc/spamassassin/trunk/.github/actions/setup_macos_for_sa_test/action.yml?rev=1905790&view=auto
==============================================================================
--- spamassassin/trunk/.github/actions/setup_macos_for_sa_test/action.yml (added)
+++ spamassassin/trunk/.github/actions/setup_macos_for_sa_test/action.yml Tue Dec 6 11:05:53 2022
@@ -0,0 +1,37 @@
+name: 'Setup macOS for SpamAssassin build and test'
+description: 'Setup macOS runner for SapmAssassin build and test'
+runs:
+ using: 'composite'
+ steps:
+ - name: remove default user group membershipp in root group
+ run: |
+ sudo dseditgroup -o edit -d runner -t user wheel
+ echo "$HOME/.local/bin" >> $GITHUB_PATH
+ shell: bash
+
+ - name: install required homebrew apps and create links for some keg-only ones, remove brew perl
+ run: |
+ brew install coreutils python3 re2c openssl berkeley-db libmaxminddb
+ brew uninstall --formula perl || echo "ignore error if not installed"
+ ln -s /usr/local/opt/berkeley-db/include/* /usr/local/include/
+ ln -s /usr/local/opt/berkeley-db/lib/* /usr/local/lib/
+ ln -s /usr/local/opt/openssl/lib/libcrypto.*dylib /usr/local/lib/
+ ln -s /usr/local/opt/openssl/lib/libssl.*dylib /usr/local/lib/
+ ln -s /usr/local/opt/openssl/include/openssl /usr/local/include/openssl
+ shell: bash
+
+ - name: install some needed packages from source that aren't in homebrew
+ run: |
+ mkdir -p ~/tmp
+ cd ~/tmp
+ wget https://www.dcc-servers.net/dcc/source/dcc.tar.Z
+ tar xf dcc.tar.Z
+ cd dcc-*
+ ./configure --disable-server --disable-dccm --disable-dccifd
+ make
+ sudo make install
+
+ pip3 install git+https://github.com/SpamExperts/pyzor.git
+
+ shell: bash
+

Added: spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/action.yml
URL: http://svn.apache.org/viewvc/spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/action.yml?rev=1905790&view=auto
==============================================================================
--- spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/action.yml (added)
+++ spamassassin/trunk/.github/actions/setup_ubuntu_for_sa_test/action.yml Tue Dec 6 11:05:53 2022
@@ -0,0 +1,113 @@
+name: 'Setup Ubunutu for SpamAssassin build and test'
+description: 'Setup Ubuntu runner for SapmAssassin build and test'
+runs:
+ using: 'composite'
+ steps:
+ - name: fix permission causing problem in perl taint mode
+ run: |
+ sudo chmod o-w /usr/local/bin
+ shell: bash
+
+ - name: free up disk space
+ run: |
+ df -H .
+ rm -f ~/.bash_profile # github installs npm here and that messes up use of ~/.profile
+ sudo apt-get purge -y '~ndotnet-'
+ sudo apt-get autoremove -y
+ sudo apt-get clean
+ sudo rm -rf /usr/local/lib/android # will release about 10 GB if you don't need Android
+ sudo rm -rf /usr/share/dotnet # will release about 20GB if you don't need .NET
+ df -H .
+ shell: bash
+
+ - name: install required apt packages - perl ones get their prerequisites needed for plenv perl installs
+ run: |
+ sudo apt-get update
+ sudo apt-get upgrade -y
+ sudo apt-get install -y \
+ apt-utils \
+ build-essential \
+ curl \
+ wget \
+ cpanminus \
+ git \
+ pyzor \
+ razor \
+ subversion \
+ libdb-dev \
+ libdbi-dev \
+ libidn11-dev \
+ libidn2-dev \
+ libmaxminddb-dev \
+ libssl-dev \
+ zlib1g-dev \
+ poppler-utils \
+ tesseract-ocr \
+ libarchive-zip-perl \
+ libberkeleydb-perl \
+ libbsd-resource-perl \
+ libdigest-sha-perl \
+ libencode-detect-perl \
+ libgeo-ip-perl \
+ libgeoip2-perl \
+ libio-compress-perl \
+ libmail-dkim-perl \
+ libmail-spf-perl \
+ libnet-patricia-perl \
+ libfile-sharedir-install-perl \
+ libtext-diff-perl \
+ libtest-exception-perl \
+ libregexp-common-perl \
+ libxml-libxml-perl \
+ libtest-pod-coverage-perl \
+ libdbd-sqlite2-perl \
+ libdbd-sqlite3-perl \
+ libdevel-cycle-perl \
+ libgeography-countries-perl \
+ libtest-perl-critic-perl \
+ libdbix-simple-perl \
+ libemail-mime-perl \
+ libemail-sender-perl \
+ libnet-idn-encode-perl \
+ libtest-file-sharedir-perl \
+ libtest-output-perl \
+ libnet-imap-simple-perl \
+ libnet-smtps-perl
+ shell: bash
+
+ - name: install some needed packages from source
+ run: |
+ mkdir -p ~/tmp
+ cd ~/tmp
+
+ wget https://www.dcc-servers.net/dcc/source/dcc.tar.Z
+ tar xf dcc.tar.Z
+ cd dcc-*
+ ./configure --disable-server --disable-dccm --disable-dccifd
+ make
+ sudo make install
+
+ # re2c. Is there a good way of getting latest instead of hardcoding version?
+ cd ~/tmp
+ wget https://github.com/skvadrik/re2c/releases/download/3.0/re2c-3.0.tar.xz
+ tar xf re2c-3.0.tar.xz
+ cd re2c-3.0
+ ./configure
+ make
+ sudo make install
+
+ razor-admin -create
+ razor-admin -register
+
+ shell: bash
+
+ - name: install plenv
+ run: |
+ cd ~/
+ git clone https://github.com/tokuhirom/plenv.git ~/.plenv
+ git clone https://github.com/tokuhirom/Perl-Build.git ~/.plenv/plugins/perl-build/
+ echo "$HOME/.local/bin" >> $GITHUB_PATH
+ echo "$HOME/.plenv/bin" >> $GITHUB_PATH
+ echo "$HOME/.plenv/shims" >> $GITHUB_PATH
+ echo "PLENV_SHELL=bash" >> $GITHUB_ENV
+ shell: bash

Added: spamassassin/trunk/.github/workflows/makefile_one_native.yml
URL: http://svn.apache.org/viewvc/spamassassin/trunk/.github/workflows/makefile_one_native.yml?rev=1905790&view=auto
==============================================================================
--- spamassassin/trunk/.github/workflows/makefile_one_native.yml (added)
+++ spamassassin/trunk/.github/workflows/makefile_one_native.yml Tue Dec 6 11:05:53 2022
@@ -0,0 +1,118 @@
+# based on https://perlmaven.com/setup-github-actions
+
+name: One Native runner CI
+
+on:
+ workflow_dispatch:
+ inputs:
+ runner:
+ description: 'Runner to use ubuntu-22.04 macos-latest or windows-latest'
+ required: true
+ default: 'macos-latest'
+ perl:
+ description: 'Perl major version 5.xx xx even 14 - 34'
+ required: true
+ default: '5.30'
+ make:
+ description: 'make command to use make gmake or dmake'
+ required: true
+ default: 'make'
+jobs:
+ perl-job:
+ env:
+ PERL5LIB: ''
+ PERL_VERSION: ${{github.event.inputs.perl}}
+ runs-on: ${{github.event.inputs.runner}}
+ name: OS ${{github.event.inputs.runner}} Perl ${{github.event.inputs.perl}}
+
+ steps:
+ - uses: actions/checkout@v2
+
+ - name: Install requirements for Ubuntu
+ if: ${{ startsWith( github.event.inputs.runner, 'ubuntu-' ) }}
+ uses: ./.github/actions/setup_ubuntu_for_sa_test
+
+ - name: Install requirements for macOS
+ if: ${{ startsWith( github.event.inputs.runner, 'macos-' ) }}
+ uses: ./.github/actions/setup_macos_for_sa_test
+
+ - name: Set up perl windows
+ if: ${{ startsWith( github.event.inputs.runner, 'windows-' ) }}
+ uses: shogo82148/actions-setup-perl@v1
+ with:
+ perl-version: ${{ github.event.inputs.perl }}
+ distribution: ${{ ( startsWith( github.event.inputs.runner, 'windows-' ) && 'strawberry' ) || 'default' }}
+
+ - name: Set up perl macos
+ if: ${{ startsWith( github.event.inputs.runner, 'macos-' ) }}
+ run: |
+ # ensure that the default perl and some related formulae are not installed before installing our custom version
+ brew uninstall -q --formula perl cpanminus cpm 2&>1 || true
+ brew install "sidney/perl/perl@${PERL_VERSION}"
+ brew link "perl@${PERL_VERSION}"
+ hash -r
+ brew install -s cpanminus
+
+ - name: Set up perl ubuntu
+ if: ${{ startsWith( github.event.inputs.runner, 'ubuntu-' ) }}
+ run: |
+ # find the latest sub version of the requested perl version that plenv has available
+ perlversion=$(plenv install --list | grep -m 1 -o "${PERL_VERSION}..")
+ plenv install "$perlversion"
+ plenv global "$perlversion"
+ hash -r
+ perl -v
+ plenv install-cpanm
+
+ - name: Install dependencies and optional modules for full testing Ubuntu and macOS
+ if: ${{ !startsWith( github.event.inputs.runner, 'windows-' ) }}
+ uses: ./.github/actions/install_cpan_dependencies_for_sa
+
+ - name: Install dependencies and optional modules for full testing on Windows
+ if: ${{ startsWith( github.event.inputs.runner, 'windows-' ) }}
+ run: |
+ cpanm --notest Module::Install Archive::Zip BSD::Resource BerkeleyDB Compress::Zlib DBI DB_File Devel::Cycle Digest::SHA Digest::SHA1 Email::Address::XS Encode::Detect Encode::Detect::Detector Geo::IP GeoIP2 GeoIP2::Database::Reader Geography::Countries HTML::Parser HTTP::Cookies HTTP::Daemon HTTP::Date HTTP::Negotiate IO::Socket::INET6 IO::Socket::SSL IO::String IP::Country IP::Country::DB_File LWP::Protocol::https LWP::UserAgent Mail::DKIM Mail::DMARC::PurePerl Math::Int128 MaxMind::DB::Reader::XS Net::CIDR::Lite Net::DNS Net::DNS::Nameserver Net::LibIDN Net::LibIDN2 Net::Patricia Net::Works::Network NetAddr::IP Params::Validate Sys::Hostname::Long Test::Perl::Critic Test::Pod Test::Pod::Coverage WWW::RobotRules Text::Diff Perl::Critic::Policy::Bangs::ProhibitBitwiseOperators Perl::Critic::Policy::Bangs::ProhibitDebuggingModules List::MoreUtils Perl::Critic::Policy::Perlsecret Perl::Critic::Policy::Compatibility::ProhibitThreeArgumentOpen Perl::Critic::Policy::Lax::Prohi
bitStringyEval::ExceptForRequire Perl::Critic::Policy::ValuesAndExpressions::PreventSQLInjection Perl::Critic::Policy::ControlStructures::ProhibitReturnInDoBlock || echo "warn: errors installing cpan modules have been ignored"
+ cpanm Razor2::Client::Agent -n -v --install-args="DESTINSTALLSITESCRIPT=$HOME/.local/bin=$HOME/.local/bin" || echo "warn: errors installing Razor2::Client::Agent have been ignored"
+ cpanm Mail::SPF -n --install-args="--install_path sbin=$HOME/.local/bin" || echo "warn: errors installing Mail::SPF have been ignored"
+ cpanm --installdeps --notest . || echo "warn: errors installing required cpan modules have been ignored"
+ echo "Finished step"
+
+ - name: Show content of log files on Windows
+ if: ${{ failure() && startsWith( github.event.inputs.runner, 'windows-' ) }}
+ run: cat C:/Users/runner/.cpanm/work/*/build.log
+
+ - name: Regular Tests Ubuntu
+ if: ${{ startsWith( github.event.inputs.runner, 'ubuntu-' ) }}
+ run: |
+ perl -v
+ perl -i -pe 's/run_(long_tests|net_tests|dcc_tests|sql_pref_tests|spamd_prefork_stress_test)=n/run_$1=y/' t/config.dist
+ perl Makefile.PL
+ ${{ github.event.inputs.make }} ENABLE_SSL=yes
+ ${{ github.event.inputs.make }} test
+
+ - name: Regular Tests macOS
+ if: ${{ startsWith( github.event.inputs.runner, 'macos-' ) }}
+ run: |
+ # This wouldn't be necessary if we could install arbitrary perl versions with homebrew, but perls we are using have an old berkeley db version
+ # maybe ok now. This checks if it is
+ perl -MIP::Country::DB_File -e 'my $foo; eval {$foo = IP::Country::DB_File->new("t/data/geodb/ipcc.db"); 1;}; if ($@) {my $err = $@; print "err is $@\n";}'
+ perl -i -pe 's/run_(long_tests|net_tests|dcc_tests|sql_pref_tests|spamd_prefork_stress_test)=n/run_$1=y/' t/config.dist
+ perl Makefile.PL
+ ${{ github.event.inputs.make }} ENABLE_SSL=yes
+ sudo -u runner ${{ github.event.inputs.make }} test
+
+ - name: Regular Tests Windows
+ if: ${{ startsWith( github.event.inputs.runner, 'windows-' ) }}
+ run: |
+ perl -i.bak -pe "s/run_(long_tests|net_tests|dcc_tests|sql_pref_tests|spamd_prefork_stress_test)=n/run_$1=y/" t/config.dist
+ perl Makefile.PL
+ ${{ github.event.inputs.make }}
+ ${{ github.event.inputs.make }} test
+ shell: cmd
+
+ - name: Publish logs as artifacts
+ if: ${{ failure() }}
+ uses: actions/upload-artifact@v3
+ with:
+ name: test_logs
+ path: t/log/*/d.*/*

Modified: spamassassin/trunk/MANIFEST.SKIP
URL: http://svn.apache.org/viewvc/spamassassin/trunk/MANIFEST.SKIP?rev=1905790&r1=1905789&r2=1905790&view=diff
==============================================================================
--- spamassassin/trunk/MANIFEST.SKIP (original)
+++ spamassassin/trunk/MANIFEST.SKIP Tue Dec 6 11:05:53 2022
@@ -22,6 +22,7 @@
\btmon\.out$
\b[Oo][Ll][Dd]$
\b[Oo][Uu][Tt]$
+^.github/
^blib/
^blibdirs$
^build/3\.\d\.\d_change_summary$