Archive for December, 2018

This script will search for any email¬† where subject starts with “Darktrace” subject and if it find any will move it to Processed folder, all other emails will be moved to backup folder and all emails in inbox will be deleted (to mark emails as processed), also it will extract email body in human readable format.


from email.message import EmailMessage
import email
import imaplib
import re
import sys
import logging
import base64
import email.parser
import html2text
import requests
import json
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-mpass', '-mailbox_password', dest = 'mailbox_password', help = 'mailbox password.')
args = parser.parse_args()

user = ''
mailbox_password = args.mailbox_password

def get_email_body(body):

       if body.is_multipart():
         for payload in body.get_payload():
             print('To:\t\t', body['To'])
             print('From:\t', body['From'])
             print('Subject:', body['Subject'])
             for part in body.walk():
               if (part.get_content_type() == 'text/plain') and (part.get('Content-Disposition') is None):
                output = part.get_payload()
         print('To:\t\t', body['To'])
         print('From:\t', body['From'])
         print('Subject:', body['Subject'])
         print('Date:\t', body['Date'])
         print('Thread-Index:\t', body['Thread-Index'])
         text = f"{body.get_payload(decode=True)}"
         html = text.replace("b'", "")
         h = html2text.HTML2Text()
         h.ignore_links = True
         output = (h.handle(f'''{html}''').replace("\\r\\n", ""))
         output = output.replace("'", "")
         # output in one line
         #output = output.replace('\n'," ")
         output = output.replace('*', "")
         return output

def clear_inbox(conn, dest_folder):
    result = conn.uid('COPY', emailid, dest_folder)
    if result[0] == 'OK':
     result = mov, data = conn.uid('STORE',emailid, '+FLAGS', '(\Deleted Items)')

conn = imaplib.IMAP4_SSL("")


  resp, items = conn.uid("search",None, 'All')
  items = items[0].split()

  for emailid in items:
   resp, data = conn.uid("fetch",emailid, "(RFC822)")
   if resp == 'OK':
     email_body = data[0][1].decode('utf-8')
     email_message = email.message_from_string(email_body)
     subject = email_message["Subject"]
     if subject.lower().startswith('Darktrace'.lower()):
         output = get_email_body(email_message)

         #do some task
         # move emails to Processed folder and clear Inbox
         clear_inbox(conn, "Processed")
        clear_inbox(conn, "backup")

except IndexError:
     print("No new email")


Second method is using BeatiufoulSoup HTML parser

from bs4 import BeautifulSoup

def print_payload(message):
    if message.is_multipart():
        for payload in message.get_payload():
         #print (message.get_payload())
         for part in message.walk():
             if part.get_content_type():
                 body = str(part.get_payload())
                 soup = BeautifulSoup(body,features="html.parser")
                 paragraphs = soup.find_all('p')
                 for paragraph in paragraphs:
                     print(paragraph.text.encode('utf-8').decode('ascii', 'ignore'))


mail = email.message_from_string(email_body)

Rundeck – Nginx as SSL reverse proxy

Posted: December 14, 2018 in RunDeck

In this post we installed Rundeck, in this one we’ll access to Rundeck by typing https://FQDN, without specifying port number is specified as name in rundeck properties before start nginx is installed.

Install nginx:

yum install nginx

Creating Self signed SSL certificate:

cd /etc/nginx
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/rundeck.key -out /etc/nginx/rundeck.crt

Generating a 2048 bit RSA private key
writing new private key to '/etc/nginx/rundeck.key'
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [XX]:SR
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:Zemoon
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []
Email Address []:

Make sure Rundeck FQDN (submitted during SSL certificate creation) is resolvable


cat /etc/nginx/conf.d/rundeck.conf
server {
listen 443 ssl;
server_name; # Replace it with your Subdomain

access_log /var/log/nginx/rundeck.yallalabs.local.access.log;

ssl_certificate /etc/nginx/rundeck.crt;
ssl_certificate_key /etc/nginx/rundeck.key;

ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;

location / {
#add_header Front-End-Https on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

proxy_read_timeout 90;

proxy_redirect; # Replace it with your Subdomain


server {
listen 80;
server_name; # Replace it with your Subdomain
return 301 https://$host$request_uri;
cat /etc/rundeck/
# -

# ----------------------------------------------------------------
# Rundeck server connection information
# ---------------------------------------------------------------- =
framework.server.hostname =
framework.server.port = 4440
framework.server.url =

# ----------------------------------------------------------------
# Installation locations
# ----------------------------------------------------------------



# ----------------------------------------------------------------
# SSH defaults for node executor and file copier
# ----------------------------------------------------------------

framework.ssh.keypath = /var/lib/rundeck/.ssh/id_rsa
framework.ssh.user = rundeck

# ssh connection timeout after a specified number of milliseconds.
# "0" value means wait forever.
framework.ssh.timeout = 0
# ----------------------------------------------------------------
# Auto generated server UUID: 391d3428-9a67-44d9-9d55-9427b52387c0
# ----------------------------------------------------------------
rundeck.server.uuid = 391d3428-9a67-44d9-9d55-9427b52387c0
cat /etc/rundeck/
#loglevel.default is the default log level for jobs: ERROR,WARN,INFO,VERBOSE,DEBUG

#rss.enabled if set to true enables RSS feeds that are public (non-authenticated)
# change hostname here
dataSource.dbCreate = update
dataSource.url = jdbc:h2:file:/var/lib/rundeck/data/rundeckdb;MVCC=true
rundeck.log4j.config.file = /etc/rundeck/

Restart nginx and rundeckd

systemctl restart nginx && systemctl restart rundeckd

If all is fine, rundecl should be accessible from https//

In this post we configured Windows instances to go to hibernation. In this one we’ll modify Node.JS script to shut down EC2 instances if no one is connected via SSH connections.

Setting SSH idle timeout settings

cat /etc/profile.d/
export TMOUT=900
readonly TMOUT



Installing SSM Agent on CentOS 7

yum install -y
systemctl status amazon-ssm-agent
systemctl enable amazon-ssm-agent
systemctl start amazon-ssm-agent

How to create SSM role and how to assign it to instance check out here. and here

Create following tags:




Change only code in auto_stop/modules/control/index.js

// Import Dependencies
let AWS = require('aws-sdk');
AWS.config.region = "eu-west-1";

module.exports.getInstanceIds = () => {
return new Promise(
(resolve, reject) => {
let ec2 = new AWS.EC2();
let params = {
Filters: [
{ Name: "instance-state-name",
Values: ["running"]
Name: "tag:Auto_Stop_Schedule",
Values: ["1"]
Name: "tag:Auto_Stop_Enabled",
Values: ["True","true", "Yes", "yes"]
Name: "tag:Auto_Stop_Type",
Values: ["Linux","linux"]

ec2.describeInstances(params, (err, data) => {
if (err) reject(err);

let instanceIds = [];
let reservations = "";
try {
reservations = data.Reservations;
catch(err) {
if(Array.isArray(reservations)) {
reservations.forEach((reservation) => {
reservation.Instances.forEach((instance) =>{
if(instanceIds.length >= 1) {
"InstanceIds": instanceIds
else {
console.log("[Info] getInstanceIds: No instances found.");
else {
reject(new Error("[Error] getInstanceIds: Reservations is not an array."));

module.exports.shutdownInstances = (controlObj) => {
return new Promise(
(resolve, reject) => {
let ssm = new AWS.SSM();

let instanceIds = controlObj.InstanceIds;
instanceIds.forEach((i) => {
let ssmParams = {
InstanceIds: [i],
DocumentName: "AWS-RunShellScript",
Parameters: {
"commands":["#!/bin/bash","LOGFOLDER=\"/var/log/ssh_check\"","LOGFILE=\"auto_stop_activity.log\"","","# Check if the ssh_check Log dir exists. If not, create it.","[ -d $LOGFOLDER ] || mkdir -p $LOGFOLDER","","if [[ \"$(/usr/bin/w | wc -l)\" -gt 2 ]];"," then"," echo \"$(date) >>> Live SSH session detected\" "," echo \"$(date) >>> Live SSH session detected\" >> \"$LOGFOLDER/$LOGFILE\""," else"," # If no active SSH sessions are found, shutdown the instance."," echo \"$(date) >>> No running sessions detected\" "," echo \"$(date) >>> Shutting down...\"",""," echo \"$(date) >>> No running sessions detected\" >> \"$LOGFOLDER/$LOGFILE\" "," echo \"$(date) >>> Shutting down...\" >> \"$LOGFOLDER/$LOGFILE\" "," shutdown -P -t 30 > /dev/null 2>&1"," exit 0"," fi"]
MaxErrors: "0",
TimeoutSeconds: 120

// Hibernate instances
ssm.sendCommand(ssmParams, function(err, data) {
if (err) {
console.log(`Error: ${err}`);
console.log("Command Sent");
console.log(`Instance to shutdown: ${instanceIds}`);


In this example 2 AD groups will be used


  • BitBucket.Admin (System Admin,Admin,Project creator,Bitbucket user)
  • BitBucket.User (Project creator,Bitbucket user)

Configuring Active Directory integration:

User directory-add directory-Microsoft Active Directory

Server Settings

Name: Active Directory server

Directory Type: Microsoft Active Directory


Port: 389

Username: ldapuser for searching AD

LDAP Schema

BaseDN: DC=example,DC=com

LDAP Permissions:

Read only

Advanced settings

Check Enable nested groups (leave other settings)

User Schema Settings

Show only members of BitBucket.Admin/Bitbucket.User AD groups

UserObject class: user

UserObjectFilter:(&(objectCategory=Person)(sAMAccountName=*)(|(memberOf:1.2.840.113556.1.4.1941:=cn=BitBucket.Admin,OU=Groups,DC=example,DC=com (memberOf:1.2.840.113556.1.4.1941:=cn=BitBucket.User,OU=Groups,DC=example,DC=com)))

UserNameAtribute: sAMAccountName

UserName RDN Atribute: cn

User First Attribute Name: givenName

User Last Name attribute: sn

User Display Name Attribute: displayName

User Email Attribute: mail

User Password Attribute: unicodePwd

User Unique Attribute: objectGUID


Group Schema settings


Group Object Class: group

Group Obect Filter: (&(objectCategory=Group)(cn=BitBucket*))

Group Name attribute: cn

Group Description Attribute: description

Configuring group access and roles

In BitBucket go to Global permissions-add groups and assign roles