Today I Learned

20 posts about #general

Persisting files between jobs in CircleCI's worklow

When you have a file or files that you want to persist between jobs in Workflow you can add the step persist_to_workspace in the job that creates the files.

For example:

Assuming that you have a job that builds JS project and creates a zip under builds/ in the workdir directory you can add step:

      - persist_to_workspace:
          root: .
          paths:
            - builds/*

And in the followup jobs you need to call

      - attach_workspace:
          at: ./

to have access to those files. The files will be accessible under ./builds directory.

Docs:

https://circleci.com/docs/2.0/configuration-reference/#persist_to_workspace https://circleci.com/docs/2.0/configuration-reference/#attach_workspace

Spotify loves space on your hard disk

Check if it is a problem for you. But usually, Spotify cache takes about 10gb of space.

SpotifyCache

How to fix that.

1) Shut down you Spotify

2) Remove whole folder com.spotify.client (also from the bin) - ~Library/Caches/com.spotify.client

3) Locate the Spotify config folder (for me it was ~/Library/Application Support/Spotify)

4) Open prefs file with your favorite text editor.

5) Add another line and put a value in megabytes right after the equals sign. I chose 1024mb.

storage.size=1024

6) Relaunch Spotify.

Publishing private repository with public dependencies

Simply using https or ssh links in package.json can backfire for people who have or didn’t have used any additional security measures like 2FA or SSH.

https: "git+https://github.com/OWNER/REPO.git" - would fail when 2FA is set up, and would fail on token authorization. ssh: "git@github.com:OWNER/REPO.git" - obviously fails when no SSH key is defined. (Its recommended to have one)

So We decided to use Github Package Registry that allows you to easily publish public/private repos (currently GPR is in open beta). All instructions how to publish/install a package are here

create personal access token for CLI

installing a package

But the latter link, can be misleading. (Although is assuming that everyone in organization has their own PAT generated, which might be a much better solution).

When you setup your .npmrc with this config:

registry=https://npm.pkg.github.com/${OWNER}/:_authToken=${PERSONAL_ACCESS_TOKEN}

during npm install it will try to download any package’s dependency from your OWNER domain even though its dependencies are public.

So you can use a config like this to avoid having public dependencies published in your domain.

${OWNER}:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${PERSONAL_ACCESS_TOKEN}

e.g

@bobrimperator:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=my-generated-token-with-read-permission

Further references: Setting up fontawesome pro in your project

Generic with class as argument and returning instance

So let’s imagine you have few classes. Let’s say it’s just something like

class Foo {};
class Bar {};
class Baz {};

You want to create a function that accepts a class as an argument and based on that, it returns class instance.

The best approach would be to have TS generics.

function create<T>(model: T): T {
    // Error!
    // This expression is not constructable.
    // Type '{}' has no construct signatures.
    return new model();
}

If we use it like that, we’ve got an error that type doesn’t have construct signatures.

Why is that?

It’s because, when you are defining some variable or agument to be type of a class, what you actually mean is that this variable needs to be instance of given class!

So how can we tell typescript that we want class constructor?

We can tell typescript that argument (or variable) needs to be class constructor using new () => Type or { new(): Type }

function create<T>(model: new () => T): T {
    return new model();
}


create(Foo); // returns Foo instance
create(Bar); // returns Bar instance
create(Baz); // returns Baz instance

However, there is one catch here! new () needs to have the same signature as class constructor, otherwise it will throw an error as well

class Foo {
    constructor(a: number) {
    }
}

create(Foo); // Error!

You can also work around it by having

new (...args: any[]) => T

…that is if you don’t care about constructor arguments much… ;)

Backup and Restore Firebase Firestore data

If you want to backup or restore data from firebase firestore, you have to:


1. Create a new bucket on Google Cloud Console

https://console.cloud.google.com
On “Location Type” step, You can only use United States regions options, Europe options not working for imports/exports, so I recommend using Multi-region with US option selected


2. Setup gcloud via Cloud Shell

https://console.cloud.google.com/?cloudshell=true
using command: 

gcloud config set project YOUR_PROJECT_NAME

3.1 Export firestore data using same Cloud Shell and command: 
gcloud alpha firestore export gs://YOUR_BUCKET_NAME

3.2. Import firestore data using same Cloud Shell:

Firstly, you probably want to change to another project than this, which You’ve backed up previously, don’t you?: 

gcloud config set project YOUR_ANOTHER_PROJECT_NAME

Then you can import data by using this command:

gcloud alpha firestore import gs://YOUR_BUCKET_NAME/FOLDER_NAME (folder which was created inside a selected bucket during export function, always named as a date e.g. 
 2019-09-16T21:13:08_32030)/

If you have any errors during import, most probably there is a problem with roles for default google service account. The fastest way to add required role is by using the command below:

gsutil iam ch serviceAccount:YOUR_PROJECT_NAME@appspot.gserviceaccount.com:roles/storage.admin \
    gs://YOUR_BUCKET_NAME

Fetching single file from private repository (+ CI)

We had a situation in which we did need to write an API for an app, but decided to keep it in separate repository and deploy as separate app. This API would use original app’s database in read-only mode. The problem was how to prepare database structure for testing purpose. We’ve decided to use structure.sql from original app, but we did want to keep it in sync somehow.

First thing was to get Github’s personal access token

Then, locally we’ve just altered bin/setup to include following code

require 'dotenv/load'
#...
puts "\n== Importing database structure =="
  Dotenv.load
  system! %{curl -H 'Authorization: token #{ENV.fetch('DEVELOPER_ACCESS_TOKEN')}' -H 'Accept: application/vnd.github.v3.raw' -O -L https://api.github.com/repos/OtherApp/other_app/contents/db/structure.sql}
  system! 'mv structure.sql db/structure.sql'

  puts "\n== Preparing database =="
  system! 'RAILS_ENV=test bin/rails db:drop db:create db:structure:load'

while for CircleCi we did need to add following entry to .circleci/config.yml

      - run:
          name: setup-db
          command: |
            curl \
              --header "Authorization: token ${DEVELOPER_ACCESS_TOKEN}" \
              --header "Accept: application/vnd.github.v3.raw" \
              --remote-name \
              --location https://api.github.com/repos/OtherApp/other_app/contents/db/structure.sql
            mv structure.sql db/structure.sql
            bundle exec rake db:create db:structure:load --trace

Obviously do not forget to ensure correct value for DEVELOPER_ACCESS_TOKEN in your .env and on CircleCI.