Percona支持RHEL/CentOS/Debian/Ubuntu等GNU/Linux發行版,在Installing Percona Server from Repositories中列出了具體支持的發行版本,但並未顯式地說明Percona Server for MySQL的每一個釋出版本(如5.6,5.7)具體支持哪些GNU/Linux發行版本。

本文通過Percona Software Downloads提取各個可供下載的Percona版本的下載頁鏈接,再通過下載頁的HTML代碼提取各版本具體支持的GNU/Linux發行版,最後通過Shell Script代碼實現。

重要Percona官方已經停止對RHEL 5Ubuntu 12.04 LTS的支持,詳細說明見Platform End of Life (EOL) Announcement for RHEL 5 and Ubuntu 12.04 LTS

RHEL 5 was EOL as of March 31st, 2017 and Ubuntu 12.04 LTS was end of life as of April 28th, 2017.

Requirement

提取每一個Percona所支持的GNU/Linux發行版本具體支持的Percona版本列表 1. Percona具體支持哪些GNU/Linux發行版; 2. 每一個GNU/Linux發行版具體支持Percona哪些版本;

## Analysis 通過分析Percona Software Downloads頁面,提取Percona目前提供下載的版本號及對應的下載頁鏈接。

curl -fsL https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{s@.* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\1@g;p}}'

# 5.7 https://www.percona.com/downloads/Percona-Server-LATEST/
# 5.6 https://www.percona.com/downloads/Percona-Server-5.6/LATEST/
# 5.5 https://www.percona.com/downloads/Percona-Server-5.5/LATEST/
# 5.1 https://www.percona.com/downloads/Percona-Server-5.1/LATEST/

通過下載頁鏈接,提取具體支持的GNU/Linux發行版。

# 5.7
curl -fsL -x socks5h://101.100.176.53:54609 https://www.percona.com/downloads/Percona-Server-LATEST/ | sed -r -n '/Select Software Platform/{s@redhat/@rhel@g;s@<\/option>@\n@g;p}' | sed -r -n 's@.*/([^"]*)"[[:space:]]*>.*@\1@g;/binary|source|select|^$/d;p'

# wheezy
# jessie
# stretch
# rhel6
# rhel7
# precise
# trusty
# xenial
# yakkety
# zesty


# 多行轉換爲一行
sed -r ':a;N;$!ba;s@\n@ @g;'

# wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty

Code Snippets

通過兩種方式實現:while循環和parallel並行操作。

Original

利用while循環,較爲耗時

#!/usr/bin/env bash

download_tool=${download_tool:-'curl -fsL'}
version_list_info=$(mktemp -t tempXXXXX.txt)

# - version|distro_list
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{s@.* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\1@g;p}}' | while read -r version url; do
    distro_list=${distro_list:-}
    distro_list=$($download_tool $url | sed -r -n '/Select Software Platform/{s@redhat/@rhel@g;s@<\/option>@\n@g;p}' | sed -r -n 's@.*/([^"]*)"[[:space:]]*>.*@\1@g;/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;s@\n@ @g;')
    echo "${version}|${distro_list}" >> "${version_list_info}"
done

# cat "${version_list_info}"
# 5.7|wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.6|wheezy jessie stretch rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.5|wheezy jessie rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.1|squeeze rhel5 rhel6 lucid precise


# - distro|version_list
sed -r 's@.*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;s@\n@ @g;s@ @\n@g' | awk '!a[$0]++' | while read -r distro; do
    version_list=$(awk -F\| 'match($NF,/[[:space:]]*rhel6[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;s@\n@ @g;')
    echo "$distro|$version_list"
done

# wheezy|5.7 5.6 5.5 5.1
# jessie|5.7 5.6 5.5 5.1
# stretch|5.7 5.6 5.5 5.1
# rhel6|5.7 5.6 5.5 5.1
# rhel7|5.7 5.6 5.5 5.1
# precise|5.7 5.6 5.5 5.1
# trusty|5.7 5.6 5.5 5.1
# xenial|5.7 5.6 5.5 5.1
# yakkety|5.7 5.6 5.5 5.1
# zesty|5.7 5.6 5.5 5.1
# rhel5|5.7 5.6 5.5 5.1
# squeeze|5.7 5.6 5.5 5.1
# lucid|5.7 5.6 5.5 5.1

[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Parallel

利用parallel命令並行操作,較節省時間

#!/usr/bin/env bash
download_tool=${download_tool:-'curl -fsL'}

version_list_info=$(mktemp -t tempXXXXX.txt)
export version_list_info="${version_list_info}"

# - version|distro_list
funcDistroListForPerVersion(){
    # $1 = 5.7 https://www.percona.com/downloads/Percona-Server-LATEST/
    local item=${1:-}
    if [[ -n "${item}" ]]; then
        local version=${version:-}
        local version_url=${version_url:-}
        local distro_list=${distro_list:-}
        version="${item%% *}"
        version_url="${item##* }"
        distro_list=$($download_tool $version_url | sed -r -n '/Select Software Platform/{s@redhat/@rhel@g;s@<\/option>@\n@g;p}' | sed -r -n 's@.*/([^"]*)"[[:space:]]*>.*@\1@g;/binary|source|select|^$/d;p' | sed -r ':a;N;$!ba;s@\n@ @g;')
        echo "${version}|${distro_list}" >> "${version_list_info}"
    fi
}

export -f funcDistroListForPerVersion
export download_tool="${download_tool}"
$download_tool https://www.percona.com/downloads/ | sed -r -n '/A drop-in replacement for MySQL/,/<\/div>/{/<a/{s@.* href="([^"]*)".*>Download ([[:digit:].]+).*@\2 https://www.percona.com\1@g;p}}' | parallel -k -j 0 funcDistroListForPerVersion 2> /dev/null

# cat "${version_list_info}"
# 5.7|wheezy jessie stretch rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.6|wheezy jessie stretch rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.5|wheezy jessie rhel5 rhel6 rhel7 precise trusty xenial yakkety zesty
# 5.1|squeeze rhel5 rhel6 lucid precise

# - distro|version_list
funcVersionListPerDistro(){
    local distro=${1:-}
    local version_list=${version_list:-}
    version_list=$(awk -F\| 'match($NF,/[[:space:]]*'"${distro}"'[[:space:]]*/){print $1}' "${version_list_info}" | sed -r ':a;N;$!ba;s@\n@ @g;')
    echo "$distro|$version_list"
}

export version_list_info="${version_list_info}"
export -f funcVersionListPerDistro
sed -r 's@.*\|@@g' "${version_list_info}" | sed -r ':a;N;$!ba;s@\n@ @g;s@ @\n@g' | awk '!a[$0]++' | parallel -k -j 0 funcVersionListPerDistro 2> /dev/null

# wheezy|5.7 5.6 5.5 5.1
# jessie|5.7 5.6 5.5 5.1
# stretch|5.7 5.6 5.5 5.1
# rhel6|5.7 5.6 5.5 5.1
# rhel7|5.7 5.6 5.5 5.1
# precise|5.7 5.6 5.5 5.1
# trusty|5.7 5.6 5.5 5.1
# xenial|5.7 5.6 5.5 5.1
# yakkety|5.7 5.6 5.5 5.1
# zesty|5.7 5.6 5.5 5.1
# rhel5|5.7 5.6 5.5 5.1
# squeeze|5.7 5.6 5.5 5.1
# lucid|5.7 5.6 5.5 5.1

[[ -f "${version_list_info}" ]] && rm -f "${version_list_info}"
unset version_list_info

Output

將Shell Script代碼段合併後得到的結果如下

Distro Percona Version
wheezy 5.7, 5.6, 5.5, 5.1
jessie 5.7, 5.6, 5.5, 5.1
stretch 5.7, 5.6, 5.5, 5.1
rhel6 5.7, 5.6, 5.5, 5.1
rhel7 5.7, 5.6, 5.5, 5.1
precise 5.7, 5.6, 5.5, 5.1
trusty 5.7, 5.6, 5.5, 5.1
xenial 5.7, 5.6, 5.5, 5.1
yakkety 5.7, 5.6, 5.5, 5.1
zesty 5.7, 5.6, 5.5, 5.1
rhel5 5.7, 5.6, 5.5, 5.1
squeeze 5.7, 5.6, 5.5, 5.1
lucid 5.7, 5.6, 5.5, 5.1

Performance Comparison

使用bash內置命令time對操作進行計時

# Original Time
real	0m4.429s
user	0m0.136s
sys	0m0.024s

# Parallel Time
real	0m2.456s
user	0m0.544s
sys	0m0.160s

可以看到使用parallel,操作耗時節省接近一半時間。

Reference

Change Logs

  • 2017.08.13 21:26 Sun Asia/Shanghai
    • 初稿完成