The article contains some useful "grep", "sed", "awk" and "jq" commands.
SED substitute part of match using variables:
sed -i 's/openstack_\(haproxy_[a-zA-Z0-9.\/?=_-]*_check_params\)/\1/' *.yml"1" is a variable it takes everything in braces
Result:
openstack_haproxy_aodh-api_check_params: check => haproxy_aodh-api_check_params: check
SED remove leading spaces:
cat ./formulas2 | sed -e 's/^[[:space:]]*//'SED remove empty lines:
cat ./test.txt | sed '/^[[:space:]]*$/d'SED remove spaces:
sed 's/[[:blank:]]//g'SED print text between two patterns:
sed -n '/# From oslo.messaging/,/transport_url/p' ./neutron.confGroup matching (look behind assertion):
echo "DH Parameters: (2048 bit)" | grep -Po 'DH Parameters: \(\K[0-9]+'More examples:
echo "{"_id":"scheme_version","_rev":"4-cad1842a7646b4497066e09c3788e724","scheme_version":1234}" | grep -Po 'scheme_version:\K[0-9]+' or
echo "{"_id":"scheme_version","_rev":"4-cad1842a7646b4497066e09c3788e724","scheme_version":1234}" | grep -Po '(?<=scheme_version:)[0-9]+'
SED display all text after match (the example below will print everything in the file after "containers:" pattern):
cat example.txt | sed -n -e '/containers:/,$p' SED insert line after match (in the example the line with 2 leading spaces will be inserted after "manifests:" pattern):
sed -i '/^manifests:/a \ \ secret_ca_bundle: false' ./values.yaml AWK compare versions in shell. Can be useful when bash or "sort -V" are not available (e.g. in some docker images):
compareVersions() {
echo $1 $2 | \
awk '{ split($1, a, ".");
split($2, b, ".");
res = -1;
for (i = 1; i <= 3; i++){
if (a[i] < b[i]) {
res =-1;
break;
} else if (a[i] > b[i]) {
res = 1;
break;
} else if (a[i] == b[i]) {
if (i == 3) {
res = 0;
break;
} else {
continue;
}
}
}
print res;
}'
}JQ iterate through keys in the dict, an example from AWS Secrets Manager command output:
aws secretsmanager get-secret-value --secret-id $ROLE-licence --region $REGION | jq -rc '.SecretString|fromjson|.[]'| while read i; do
echo $i >> $ARTIFACTORY_LICENSE_FILE
echo "" >> $ARTIFACTORY_LICENSE_FILE
done