Converting CSS color notations

In CSS, color can be specified using different models and notations. For RGB color model, there are
hexadecimal and RGB notations. There are other models and notations, but for this post the conversion between
notations of hexadecimal and RGB will be focused on. To carry out this conversion, a small utility will be
written in Typescript with Deno runtime. This utility will operate on stdin and write the results to stdout.
While writing this utility, topics of streams, and standard input/output will be briefly explored.

## RGB Color Model

The RGB color model defines colors by their red, green and blue values, and optionally alpha (opacity).
Depending on notation, color values are expressed in different ways. In **hexadecimal** notation values are
written in hexadecimal characters, and it can take the following forms: _#RGB_, _#RGBA_, _#RRGGBB_, _#RRGGBBAA_.
Whereas in **RGB** notation, color values are integers, each between 0 and 255. And the optional alpha value is
either between 0 and 1 or 0% and 100%. It takes the following forms: _rgb(R G B)_, _rgb(R G B / A)_, _rgb(R_, G, B),
_rgb(R, G, B, A)_.

## Using Regular Expressions

Regular expressions (regexes) are a way to describe a set of strings. Here they are going to be used to
describe almost possible way the hexadecimal and RGB color notations can be written.

Using regexes red, green, blue and alpha values contained in a color notation can be
extracted. However, as there are many ways to express color in either notation, the regexes would probably get
complicated. In simplest form, regexes that match whole color code could look like:

```custom-warning
You may have noticed that these regexes would also match incorrectly written color code! This is on purpose,
and for this program it's assumed that given color codes are correct.
```

To match hexadecimal notations: _#RGB_, _#RGBA,_ _#RRGGBB_ and _#RRGGBBAA_.

```regex:{"copyable":true}
/#[a-f\d]{3,8}/gi
```

To match RGB notations: _rgb(R,G,B)_, _rgb(R G B)_, _rgb(R, G, B, A)_, _rgb(R G B / A)_, _rgb(R, G, B, A%)_,
_rgb(R G B / A%)_ and _rgba()_ versions of these.

```regex:{"copyable":true}
/rgba?$(?:\d+[,\s]*){3}(?:[,/\s]+\d?[.]?\d+%?)?$/g
```

Now suppose these regexes are used to grab the whole color code in the block of text. Further processing will
be required for extracting individual red, green, blue and alpha values of these grabbed colors. I preferred
to use regexes to extract these values too, as they can simplify the process.

```custom-info
Regular expressions can be confusing to write, so tools like [RegExr](https://regexr.com/) are very helpful.
This tool can be used to test and explain the things you don't know about them.
```

Following regexes are used to extract red, green, blue and alpha values. They work with forementioned
different expressions of notations. Though for hexadecimal notation, it meant to process only the long form of
the notation. Hexadecimal colors will be converted to their long form before they are parsed by the regex.

For hexadecimal notation:

```regex:{"copyable":true}
/#([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})?/i
```

For rgb notation:

```regex:{"copyable":true}
/rgba?$(\d{1,3})[,\s]+(\d{1,3})[,\s]+(\d{1,3})(?:[,\/\s]+(\d?[.]?\d+%?))?$/
```

Functions below are for converting notations from one to another.

For hexadecimal to rgb conversion:

```typescript:{"copyable":true}
function hexToRgb(match: string): string {
  // transform to long form i.e from  #RGB or #RGBA to #RRGGBB or #RRGGBBAA respectively
  if (match.length <= 5) {
    match =
      '#' +
      match
        .slice(1)
        .split('')
        .reduce((w, c) => w + c + c, '');
  }

  let [_, r, g, b, a] = HEX_INNER.exec(match)!;

  // transform hexadecimal color values to decimal and merge
  const rgb = [r, g, b].map((c) => parseInt(c, 16)).join(', ');

  if (a) {
    // transform alpha from [0x00, 0xFF] range to [0, 1] range
    a = (parseInt(a, 16) / 255).toFixed(2);
  }

  return a ? `rgb(${rgb}, ${a})` : `rgb(${rgb})`;
}
```

```custom-info
Since the release of [color-level-4](https://www.w3.org/TR/css-color-4/#funcdef-rgb) specification, `rgb()`
accepts 4th parameter to add transparency, without the need for `rgba()`.
```

For rgb to hexadecimal conversion:

```typescript:{"copyable":true}
function rgbToHex(match: string): string {
  let [_, r, g, b, a] = RGB_INNER.exec(match)!;

  // transform decimal color values to hexadecimal long form and merge
  const rgb = [r, g, b].map((x) => parseInt(x).toString(16).padStart(2, '0')).join('');

  if (a) {
    // transform alpha from [0, 1] or [0%, 100%] range to [0x00, 0xFF] range (long form)
    a = Math.ceil((parseFloat(a) / (a.at(-1) === '%' ? 100 : 1)) * 255)
      .toString(16)
      .padStart(2, '0');
  }

  return `#${rgb}${a ?? ''}`;
}
```

The functions above are invoked in the following function by being passed to `convert` parameter. And the
`whole` parameter is for capturing the whole color code which is to be transformed.

```typescript:{"copyable":true}
function replace(chunk: string, whole: RegExp, convert: (s: string) => string): string {
  let local = chunk;

  // Get all color notations matching to the `whole` regex
  const matches = chunk.match(whole) ?? [];

  for (const match of matches) {
    // Replace matching color notations with desired notation
    local = local.replace(match, convert(match));
  }

  return local;
}
```

From the perspective of this function, the `chunk` parameter is a string of arbitrary length (some block of
text). This function will be used within a transform stream.

## Streams

Streams describe the data structure for flow of data. They can be used for reading, writing or transforming
data. The way these operations are done is quite different from other methods. Streams operate on data in
chunks one at a time, meaning it's not required to load or process entire data all at once. Temporary memory
used to store these chunks is called a buffer. Buffers exist until they are consumed. File, network
connections, etc. can be operated on using streams.

```custom-warning
Mind that I/O models of Node and Deno are different. Node's standard streams are not to be confused with
Deno's stream.
```

Deno aims to adhere closely to the Web Platform APIs. And It has full support for [web streams](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API).
Deno supports three kinds of streams: _Writable_, _Readable_, _Transform_. These streams are available in Deno
core runtime, can be accessed directly from global scope.

- **Writable** - Respresents destination for streaming data from which data can be written to. Accepts _underlying
  sink_ object as optional argument, which can be used to overwrite `write` method for implementation. Ex.

```typescript:{"copyable":true}
const w = new WritableStream({
  write(chunk) {
    // consume chunk
    console.log(chunk);
  },
});

// pushing chunk to be consumed
await w
  .getWriter() // WritableStreamDefaultWriter
  .write('chunk'); // Provides access to push chunk to stream queue
```

- **Readable** - Represent source of streaming data from which data can be read. Accepts _underlying source_ object
  as optional argument, which can be used to overwrite `start` method for implementation. Ex.

```typescript:{"copyable":true}
const r = new ReadableStream({
  start(controller) {
    // pushing chunk to be consumed
    controller.enqueue('chunk');
  },
});

// consume chunk
const chunk = await r
  .getReader() // ReadableStreamDefaultReader
  .read(); // Provides access to consume chunk in stream queue
```

- **Transform** - Represents both source of and destination to streaming data from which data is transformed as
  it's written and read. Accepts transformer object as optional argument, which can be used to overwrite
  `transform` method for implementation. Ex.

```typescript:{"copyable":true}
const t = new TransformStream({
  transform(chunk, controller) {
    // transforming chunk and forwarding it
    controller.enqueue(chunk);
    // not actually transforming anything just forwarding with out any change
    // which is known as `identity transform stream`
  },
});

// pushing chunk
t.writable // WritableStream
  .getWriter().write('chunk'),

// consuming 'transformed' chunk
const chunk = await t.readable // ReadableStream
                      .getReader().read(),
```

Using streams also makes it straightforward to apply composition. Streams can be connected with pipes, each
receiving data from the previous stream and passing data to the next stream. And where each stream can have a
single task to do. And all together, complex transformations can be carried out. In _Deno_, modules implementing
readable stream interface has `pipeTo` and `pipeThrough` methods. These methods can be used to connect streams
to each other. `pipeTo` accepts writable stream as argument, whereas `pipeThrough` accepts transform streams
as argument. A group of streams connected by pipes is called a pipe chain.

There are lot more to these standard web streams, but for purposes of this post this brief introduction is
enough.

## Standard Input/Output

From the perspective of this program, we are going to read data from `stdin`, and manipulate it, then write to
result to the `stdout`.

In Unix systems every process is initialized with three open _file descriptors_, and those correspond to
stdin(file descriptor 0), stdout(file descriptor 1), and stderr(file descriptor 2). These three file
descriptors implements stream interface. stdin is a readable stream, stdout is a writable stream, and stderr
is a writable stream. Diagram below illustrates streams and pipes used in this program.

```
stdin -> Text Decode | Transform Notations | Text Encode -> stdout
```

Data here flows from left to right. `|` symbol denotes a pipe. Read from stdin, decoded(raw data to string),
manipulated(notations are exchanged), encoded(string to raw data), and then written to stdout. Text Decoder
converts raw data(ex. Uint8Array) to string, and Text Encoder converts string to raw data(ex. Uint8Array).

The text is decoded because when data is read from stdin, it's of type `Uint8Array`. And similarly when data
is written to stdout, it has to be encoded to type `Uint8Array`. And classes for Text Encoder Stream and Text
Decoder Stream are available in global scope in Deno runtime.

A Transform Stream can be implementing by suppliying a `transform` method to transformer object in instance
creation. And its usage in this program looks like below:

```typescript:{"copyable":true}
function converter(whole: RegExp, convert: (s: string) => string) {
  // ReadableStream
  const stdin = Deno.stdin.readable;
  // WritableStream
  const stdout = Deno.stdout.writable;

  const textDecorderStream = new TextDecoderStream();
  const textEncoderStream = new TextEncoderStream();

  const converterStream = new TransformStream({
    transform: (chunk, controller) => {
      controller.enqueue(replace(chunk, whole, convert));
    },
  });

  stdin
    .pipeThrough(textDecorderStream)
    .pipeThrough(converterStream)
    .pipeThrough(textEncoderStream)
    .pipeTo(stdout);
}
```

`convert` parameter denotes the conversion to be applied, and `whole` parameter denotes which parts of chunk
will be transformed. It could be invoked as:

```typescript
converter(HEX_WHOLE, hexToRgb);
// OR
converter(RGB_WHOLE, rgbToHex);
```

## Deno

I chose to use Deno for this program, because I was interested in seeing how different it is from Node. Deno
has code formatter, linter, installer, text runner, bundler, compiler, etc. all built in to it. It supports
TypeScript out of the box. It contains all of its core API under Deno global variable, and standard web APIs
are directly accessible in its global scope. For module import and export, it supports ES Modules syntax.
Libraries can be directly imported via URLs. Unless explicitly enabled using CLI flags, access to file system,
network interface, or environment is not permitted. And you can read more on its [docs](https://deno.land/).

Whole code is on repository [color-convert](https://github.com/kuzb/color-convert). Basic CLI functionality
has been added. And using Deno it can be directly run as:

```bash:{"copyable":true}
deno run https://raw.githubusercontent.com/kuzb/color-convert/master/mod.ts --hex < style.css
```

Or better, it could be installed. By that I mean deno creates shell script on directory `$HOME/.deno/bin`, that
runs the deno script.

```bash:{"copyable":true}
deno install https://raw.githubusercontent.com/kuzb/color-convert/master/mod.ts
```

```custom-info
Deno install root can be changed using `DENO_INSTALL_ROOT` environment variable. By default install root is at
`$HOME/.deno`. To run scripts, installed via `deno install`, `$HOME/.deno/bin` has to be added to path.
```

Or even, it could be compiled. It creates a self contained executable inside current directory. However mind
that this executable is quiet large. For me, this was 75M.

```bash:{"copyable":true}
deno compile https://raw.githubusercontent.com/kuzb/color-convert/master/mod.ts color-convert
```

Given program can be run as standalone script/executable:

```bash:{"copyable":true}
color-convert --rgb < style.css
```

Same file can be read and written:

```bash:{"copyable":true}
color-convert --rgb < style.css > out.css && mv out.css style.css
```

Same file can't be read and written simultaneously, that's why an intermediate file is used. Or utility
_sponge_ from [moreutils](https://joeyh.name/code/moreutils/) can be used to write to same file. This utility
achieves this by waiting for the termination of the command preceding it, and hold the contents in buffer till
that.

```bash:{"copyable":true}
color-convert --rgb < style.css | sponge style.css
```

Programs that operate on standard input/output can be quite versatile. For example, this program can be ran
inside vim, over range of text.

![neovim](vim.gif)


Caching Git Credentials on Remote Hosts

I own several Raspberry Pi's. I have been recently tinkering with Ansible to ease the process of setting up
them, and thereby also learning it. One thing I want to set up is an easy Git flow. That is passwordless
authentication, caching of credentials, etc. On my daily drive, I use public key authentication with SSH.
However, for these machines (remote hosts), I am considering a simpler method. I have used personal access
token(PAT) before. And I think that using PAT could be more straight forward way.

PAT can be used in conjunction with GitHub CLI. So GitHub CLI will be downloaded and installed to the hosts,
then PAT will be used to authenticate to GitHub, and finally GitHub CLI will be used as Git Credential Helper.
To accomplish this task, an Ansible Playbook will be created.

## Ansible

[Docs](https://docs.ansible.com/) defines the function of _Ansible_ as “automates the management of remote
systems and controls their desired state.” Having Python installed and SSH configured on remote systems are
only requirements for Ansible to control and manage them. Example usages include setting up a web server or
deploying an application. These usages can be expressed as the desired state remote hosts should reach. A file
written in YAML is used to verbalize this desired state declaratively, then Ansible uses this file to move
remote hosts to the desired state.

Ansible ran from **Control Node** (local host) to manage and control **Managed Node** (remote host). A list of
logically organized remote hosts is stored in an **Inventory** file. The Inventory file is implicitly referred
inside **Playbook** to assign mappings between remote hosts and roles/tasks. **Role** implies a behavior to be
implemented, or a desired state to be reached. Roles achieve this by using tasks, handlers, variables,
plugins, etc. **Task** is a combination of action (a module and its arguments) and logical statements like
loops, and conditionals. A **Module** is a unit of code (or binary) that Ansible copies to and runs on remote
hosts.

Example:

```
Playbook: Some files should exist on certain hosts.
Role: Files should exist.
Task: Copy a file from a local host to remote host. 
Action: Use copy module with file source and destination as its arguments.
```

Roles, Playbooks, Variables, etc. can be stored separately in their own files.

```
├── inventory.yml
├── playbook.yml
├── roles
│   ├── gh
│   │   ├── defaults
│   │   │   └── main.yml
│   │   └── tasks
│   │        └── main.yml
```

However, in this post, we are going to combine everything in a single playbook file to achieve the
forementioned goal. This playbook file will contain variables and an ordered list of tasks.

I attempted to provide an overview of Ansible, though there may be additional details I have missed. Nevertheless,
it should be sufficient for following the rest of the post.

## Installing GitHub CLI

Here steps listed on [GitHub CLI docs](https://github.com/cli/cli/blob/trunk/docs/install_linux.md) will be
reproduced for Ansible. I am targeting **Debian** based machines, but still it's straight forward to adapt
it to other distributions.

Let's add the GitHub CLI software repository(package source of GitHub CLI for Apt) to Apt sources.
Variables seen inside double braces will be added at last section where whole Playbook is created.

```custom-info
`Set machine architecture` task writes the result of `dpkg --print-architecture` command to
`dpkg_architecture` variable which is then registered as fact. Facts are global variables that contains
information about remote hosts, where facts include information on operation system, architecture, etc.
```

Double braces inside strings tells templating engine used by Ansible to print value it contains, thereby
injecting it to string. These four separate blocks are **tasks**, and these tasks performing **actions** using
**modules** such as _get_url_ and _shell_. For example the task using module _get_url_ invokes it with
**arguments**: _url_, _dest_, and _mode_.

```yaml:{"copyable":true}
- name: Copy GitHub CLI Archive Keyring
  become: yes
  get_url:
    url: '{{ github_cli_keyring_url }}'
    dest: '{{ github_cli_keyring_file }}'
    mode: '0644'

- name: Set machine architecture
  shell: dpkg --print-architecture
  register: dpkg_architecture

- set_fact:
    dpkg_architecture: '{{ dpkg_architecture.stdout }}'

- name: Add GitHub CLI repository into sources list
  become: yes
  apt_repository:
    repo: 'deb [arch={{ dpkg_architecture }} signed-by={{ github_cli_keyring_file }}] https://cli.github.com/packages stable main'
    state: present
    filename: github-cli
```

Now it's available to install via the apt package manager. But before that package lists have to be updated.

```yaml:{"copyable":true}
- name: Update package list
  become: yes
  ignore_errors: yes
  apt:
    update_cache: yes

- name: Ensure latest GitHub CLI is installed
  become: yes
  apt:
    name: gh
    state: latest
```

## Authentication to GitHub

```custom-warning
Mind that GitHub CLI will automatically store the credentials when HTTPS used as preferred protocol for Git
operations.
```

PAT creation steps can be followed from
[here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token).
I prefer to use a single PAT for all the remote hosts. Now the created token has to be copied/send over to
hosts, and then the GitHub authentication process can be initiated. Before copying/sending it over, it has to
be stored inside a medium such as file, environment variables, or Ansible vault on the local host. Normally
it's recommended to use the vault for tokens, secrets and such. Depending on how it has chosen to be stored,
different operations have to be done while transferring to remote hosts. Here in this section, only options of
using environment variables and files are going to be explored.

```custom-info
It's not actually copied or send over to remote hosts in traditional sense. GitHub CLI authentication command
is expecting the token to be provided via standard input. Ansible going to execute this command on remote
hosts using `shell` module with `stdin` as the argument. No matter which ever method is used to store the
token, the value written to `stdin` will be provided by the local host. Therefore, before this
authentication command is executed on remote hosts, the value for the token will be sent over the network
along with the command where it injected to `stdin` of the command.
```

### Using environment variable

Login to GitHub via PAT using environment variables. Here _Lookup_ plugin is used to access for environment
variable on the local host is used. Lookup plugins are executed and evaluated on local host. Lookup plugins
can be used to retrieve data from local host.

```yaml:{"copyable":true}
- name: Authenticate to GitHub
  shell: gh auth login --git-protocol https --with-token
  args:
    stdin: "{{ lookup('env', 'PAT') }}"
```

In order to use the environment variables on our host machine within Ansible, we either want to export it or
define it before executing `ansible-playbook` or `ansible` command.

```bash
export PAT=TOKEN_FROM_GITHUB
ansible-playbook

# OR

PAT=TOKEN_FROM_GITHUB ansible-playbook
```

```custom-warning
If you add space before commands, they are not going to be added to shell history.
```

### Using file

Typically, storing tokens in plaintext might be a concern for some. However, I think it's not much different
from having plaintext SSH keys stored locally. Moreover, GitHub CLI itself caches the PAT inside a file
located in `$HOME/.config/gh/hosts.yml`, if you opt in to cache your credentials.

Here's the process of achieving this. For this task, it's assumed that the file containing the token is
located in the user's home directory. In order to expose the contents of the token file to the task, first the
_file path_ containing the token will be stored in a _variable_, then that _variable_ will be used to access
the _file contents_.

```yaml:{"copyable":true}
- name: Authenticate to GitHub
  shell: gh auth login --git-protocol https --with-token
  args:
    stdin: "{{ lookup('file', token_file) }}"
  vars:
    token_file: "{{ lookup('env', 'HOME') }}/PATH/TO/TOKEN"
```

## Use GitHub CLI as Git Credential Helper

Following task adds lines into your `.gitconfig` to make Git use GitHub CLI for Git credentials.

Those lines are:

```config:{"filename":".gitconfig"}
[credential "https://github.com/"]
  helper =
  helper = !/usr/local/bin/gh auth git-credential
[credential "https://gist.github.com/"]
  helper =
  helper = !/usr/local/bin/gh auth git-credential
```

```custom-warning
`gh auth git-credential` is command that is designed to be only used by a Git process.
```

Here is the task for achieving this:

```yaml:{"copyable":true}
- name: GitHub CLI as Git Credential Helper
  shell: gh auth setup-git
```

## Only, if not authenticated, authenticate

**Idempotency** refers to having performing a task either once or multiple times will produce the same
outcome. For this playbook so far, it's mostly true. However, most modules in Ansible achieve this by checking
whether the desired state has already been reached. If it has, they stop without performing further actions.
For example, package module will not try to install a package if it's already installed.

Even though currently harmless, running some of these tasks unnecessary can be prevented by putting guards
that check whether desired state has been already reached. One of them being "Don't try to reauthenticate, if
already authenticated".

This is done by checking the authentication status and if that fails then try to authenticate. By using
**Blocks**, logical groupings of tasks are created, which in turn allows for the implementation of exception
handling. If the task in **Block** group fails, an exception is raised, and that exception can be handled by tasks
in **Rescue** group.

```yaml:{"copyable":true}
- name: Ensure Authentication to GitHub
  block:
    - name: Check GitHub Authenticatino Status
      shell: gh auth status
  rescue:
    - name: Authenticate to GitHub
      shell: gh auth login --git-protocol https --with-token
      args:
        stdin: "{{ lookup('env', 'PAT') }}"
```

This task runs authentication action if authentication status fails. There are other ways to achieve something
similar, such as registering a variable on authentication status failure, and then running the authentication
task only when the variable is set.

## All put to together as Playbook

Although when installed, Ansible creates a default inventory on the local host, we can provide a per-project
inventory file. Typically, inventory files will have `.ini` extension, but I prefer to have them in YAML.

Example Inventory file:

```yaml:{"filename":"inventory.yml","copyable":true}
---
all:
  hosts:
    server:
      ansible_connection: ssh
      ansible_host: '192.168.1.90'
      ansible_port: 22
      ansible_user: john
      ansible_ssh_private_key_file: "{{ lookup('env', 'HOME') }}/.ssh/john"
```

All tasks in scope of the **Play** will have access to variables in the _vars_ section of the Playbook.
Here is the final version of collection of tasks inside a playbook along with variables:

```yaml:{"filename":"playbook.yml","copyable":true}
---
- hosts: all
  vars:
    github_cli_keyring_file: /usr/share/keyrings/githubcli-archive-keyring.gpg
    github_cli_keyring_url: https://cli.github.com/packages/githubcli-archive-keyring.gpg

  tasks:
    - name: Github CLI archive keyring exists
      become: true
      get_url:
        url: '{{ github_cli_keyring_url }}'
        dest: '{{ github_cli_keyring_file }}'
        mode: '0644'

    - name: Set machine architecture
      shell: dpkg --print-architecture
      register: dpkg_architecture

    - set_fact:
        dpkg_architecture: '{{ dpkg_architecture.stdout }}'

    - name: Github CLI repository is in sources list
      become: true
      apt_repository:
        repo: 'deb [arch={{ dpkg_architecture }} signed-by={{ github_cli_keyring_file }}] https://cli.github.com/packages stable main'
        state: present
        filename: github-cli

    - name: Package list updated
      become: true
      apt:
        update_cache: true

    - name: Latest Github CLI installed
      become: true
      apt:
        name: gh
        state: latest

    - name: Ensure Authentication to Github
      block:
        - name: Check Github Authentication Status
          shell: gh auth status
      rescue:
        - name: Authenticate to Github
          shell: gh auth login --git-protocol https --with-token
          args:
            stdin: "{{ lookup('env', 'PAT') }}"

    - name: Github CLI as Git Credential Helper
      shell: gh auth setup-git
```

Finally this Playbook can be ran as:

```bash:{"copyable":true}
ansible-playbook playbook.yml --inventory inventory.yml
```
