Team Platform Engineering

Sessions with team to completely transform our way of working.

Every extension is also an amputation

Marshall McLuhan coined this phrase. The gist is that while technological innovation extends human capability, it reduces other faculties of human skills.

Every extension is also an amputation...

The other day I didn't had the luxury of Google Maps, and I was amazed to find out that I had hard time navigating routes that should have been fimiliar to me.

What I suggest you read...

My suggested reading list:

• Accelerate: A must-read on building high-performing teams.
• Measure What Matters: The core of OKRs and how to implement them effectively.
• The Lean Startup: A guide to building the right product.
• Software Architecture: The Hard Parts: A solid reference for tackling architectural challenges.

Shariq Mustaquim’s books on Goodreads:

https://www.goodreads.com/review/list/128132142-shariq-mustaquim?shelf=%23ALL%23

Tough week...

This week has been incredibly challenging for everyone. To those facing redundancy or still at risk, my heart goes out to you. Please don't hesitate to reach out if you need support or just want to talk. It's clear that these changes are affecting all of us in one way or another, so it's crucial to take care of yourselves and support each other. Everyone responds to difficult situations differently, and that’s perfectly okay. It's also okay not to feel okay :(

In times like these, especially in the startup world, it's easy to feel pressured to work harder, take on more, and stretch beyond your limits. With the constant talk of "efficiency" and "doing more with less" in the broader tech industry, these feelings can intensify, leading us down a path toward burnout. To avoid this, it's important to set healthy work boundaries. This starts with having an open conversation with your manager about what you can realistically achieve and aligning on expectations. Additionally, clear communication about your work hours, especially given the multiple time zones we operate in, is key to maintaining balance and prioritizing your well-being. Setting these boundaries fosters a respectful and supportive work environment, particularly during uncertain times.

Joined Tarabut Gateway

A leading Open Finance company headquartered in Bahrain, with offices in UAE, UK and Saudi.

Using helm-secrets

Helm secrets is a great plugin to avoid checking in secrets in your Source code.

Here, I am using Hashicorp vault to store secrets and retrieve them safely in helm values files while installing helm charts.

Installation

$ helm plugin install https://github.com/jkroepke/helm-secrets

Setup

  $ export VAULT_TOKEN="s.VAULT_TOKENEXAMPLEASLDKASKDASDA" 
  $ export VAULT_ADDR="https://vault.example.com" 
  $ export HELM_SECRETS_DRIVER=vault

In vault, add the secrets:

In your helm values file, refer to the secret as follows:

db:
  db:
  database:     !vault secret/misp#db_database
  username:     !vault secret/misp#db_username
  pasword:      !vault secret/misp#db_password
  rootpasword:  !vault secret/misp#db_rootpasword

Now change the helm upgrade command as follows:

$ helm secrets upgrade misp ./helm/misp --install --wait --atomic  --namespace=misp --create-namespace  --values=./helm/misp/values.yaml

The secrets plugin will fetch and update the vault references in values file before invoking the upgrade command on helm.

Note:

To check the result of decoding, you can use:

$ helm secrets dec helm/misp/values.yaml

This will result in vaules.yaml.dec with actual decoded values from Hasicorp Vault.

GitHub action with matrix (parallel) jobs for each of the item in a JSON file

I have a file

deploy-list.json:

{
    "apiList": [
      "abc",
      "def"    
    ]
}

I would like to run a GitHub action with matrix (parallel) jobs for each of the item in the apiList above.

name: API Build

on: 
  workflow_dispatch:
    inputs:
      branch:
        description: 'Github Release Branch Name'
        required: true
        default: 'release/1.5.3'

  push:
    branches:
    - 'develop'

jobs:

  build-matrix:
    runs-on: ubuntu-latest

    steps:

      - name: Checkout Branch
        uses: actions/checkout@v2
        with:          
          ref: "${{ github.event.inputs.branch }}"

      - name: Set Matrix
        id: set-matrix
        run: echo "::set-output name=api_matrix::$(cat deploy-list.json | jq -c '.apiList')"

    outputs:
      api_matrix: ${{ steps.set-matrix.outputs.api_matrix }}

  build-api:
    environment: PRD
    needs: build-matrix
    runs-on: ubuntu-latest
    strategy:
      matrix:
        api_name: ${{ fromJson(needs.build-matrix.outputs.api_matrix) }}
    steps:
      - name: Checkout Branch
        uses: actions/checkout@v2
        with:
          ref: "${{ github.event.inputs.branch }}"

      - name: Building ${{ matrix.api_name }}
        run: |
          echo ${{ matrix.api_name }}
          # Do something here

Two parallel jobs will be invoked for "abc" and "def" apis making the workflow complete in twice as fast build time. I also using maven caching to speed up the builds even further.

Preventing secret leaks

I have added secrets to git repositories more than once in my lifetime.Once I even did it in a public repo.

One way to avoid this is to use a tool to detect secrets in the source code. This will come in the category of SAST tooling.

$ brew install gitleaks
$ brew install pre-commit
$ cd /path/to/repo
$ curl https://raw.githubusercontent.com/giantswarm/apiextensions/master/.gitleaks.toml -o .gitleaks.toml
$ git add .gitleaks.toml
$ git commit .gitleaks.toml -m "Adding .gitleaks.toml"

// Run scan
$ gitleaks detect --config=.gitleaks.toml