Today I Learned

Backup and Restore Firebase Firestore data

If you want to backup or restore data from firebase firestore, you have to:


1. Create a new bucket on Google Cloud Console

https://console.cloud.google.com
On “Location Type” step, You can only use United States regions options, Europe options not working for imports/exports, so I recommend using Multi-region with US option selected


2. Setup gcloud via Cloud Shell

https://console.cloud.google.com/?cloudshell=true
using command:

gcloud config set project YOUR_PROJECT_NAME

3.1 Export firestore data using same Cloud Shell and command:
gcloud alpha firestore export gs://YOUR_BUCKET_NAME

3.2. Import firestore data using same Cloud Shell:

Firstly, you probably want to change to another project than this, which You’ve backed up previously, don’t you?:

gcloud config set project YOUR_ANOTHER_PROJECT_NAME

Then you can import data by using this command:

gcloud alpha firestore import gs://YOUR_BUCKET_NAME/FOLDER_NAME (folder which was created inside a selected bucket during export function, always named as a date e.g. 
 2019-09-16T21:13:08_32030)/

If you have any errors during import, most probably there is a problem with roles for default google service account. The fastest way to add required role is by using the command below:

gsutil iam ch serviceAccount:YOUR_PROJECT_NAME@appspot.gserviceaccount.com:roles/storage.admin \
    gs://YOUR_BUCKET_NAME

Using Light Sensor in Google Chrome

Light Sensor needs to be enabled in Chrome: go to: chrome://flags/#enable-generic-sensor-extra-classes and enable it.

Then you will be able to get sensor readings

const sensor = new AmbientLightSensor();
sensor.start();
sensor.addEventListener('reading', () => {
  console.log(sensor.illuminance);
});

There are also activate and error events on the sensor object that can be listened to.

Full article: https://blog.arnellebalane.com/using-the-ambient-light-sensor-api-to-add-brightness-sensitive-dark-mode-to-my-website-82223e754630

Test multi-promise function without return

The tested function:

_setDevices = () => {
  const { selectCamera } = this.props

  WebRTCDevices.getPermissionForDevices({ video: true })
    .then(() => WebRTCDevices.getDevices(['cameraType']))
    .then(devices => {
      const chosenDeviceId = devices.length ? devices[0].deviceId : null
      selectCamera(chosenDeviceId)
    })
}

WebRTCDevices.getPermissionForDevices and WebRTCDevices.getDevices both return promises.

I want to test that selectCamera prop is called with the correct argument.

If we mock return values of those functions with the externally resolvable promise we can test it as follows:

it('calls select camera prop', done => {
  let resolveDevicePermissionPromise
  const permissionPromise = new Promise(resolve => {
    resolveDevicePermissionPromise = resolve
  })
  let resolveGetDevicesPromise
  const getDevicesPromise = new Promise(resolve => {
    resolveGetDevicesPromise = () => {
      resolve([{ deviceId: '1111' }])
    }
  })
  props.selectCamera = jest.fn()
  jest.spyOn(WebRTCDevices, 'getPermissionForDevices').mockReturnValue(permissionPromise)
  jest.spyOn(WebRTCDevices, 'getDevices').mockReturnValue(getDevicesPromise)
  const { component } = renderWrapper()

  component._setDevices()
  resolveDevicePermissionPromise()
  resolveGetDevicesPromise()

  // Move our expect to the end of queue, so "thens" from `component._setDevices()` execute before
  setTimeout(() => {
    getDevicesPromise.then(() => {
      expect(props.selectCamera).toHaveBeenCalledWith('1111')
      jest.resetAllMocks()
      done()
    })
  }, 0)
})

Waiting for content to be expanded in Cypress tests

When testing in Cypress, sometimes there are cases that content within accordion needs to be verified. The challenge is that sometimes, Cypress displays an error that an element is not yet visible, but it is on the page. A simple solution could be to wait for the element to be visible:

cy.contains('Text within the accordion')
  .should('be.visible');

One downside of this approach is that if there are more elements in the accordion, the last one should be check for visibility. Otherwise, Cypress will not wait for expanding the whole accordion and will proceed further with tests, e.g. trying to click an element that is not visible and hidden within the accordion.

An alternative approach to resolve the issue with the expanded accordions is to check its height and wait till it reaches a certain height:

cy.getByTestId('the-accordion')
  .invoke('height')
  .should('be.gt', 700);

Debug prod app with logpoints & conditional breakpoints

Recently we were able only to use breakpoints to inspect code execution in the production application. But if the code in which we put breakpoint is triggered multiple times we have to resume the execution. To prevent that, we can use conditional breakpoints https://developers.google.com/web/tools/chrome-devtools/javascript/breakpoints#conditional-loc

In cases when we do not want to break the execution of the code but just log some value we can use logpoints. https://developers.google.com/web/updates/2019/01/devtools#logpoints

Validate privateKey, certificate and CSR files

$ openssl pkey -in privateKey -pubout -outform pem | sha256sum
# => f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2  -
$ openssl x509 -in certificate -pubkey -noout -outform pem | sha256sum 
# => f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2  -
openssl req -in CSR -pubkey -noout -outform pem | sha256sum
# => f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2  -

When all pairs match, you can be sure that you have the right set of files for your domain.

Arel basic use case

veg = Arel::Table.new(:vegetables)

query = veg[:created_at].gteq( 5.days.ago ).and(
  veg[:color].eq("green").or(
    veg[:gardener].eq("Susan")
  )
)

query.to_sql
#  "vegetables"."created_at" >= '2016-12-13 03:54:28.575342'
#    AND ("vegetables"."color" = 'green' OR "vegetables"."gardener" = 'Susan')

Vegetable.where( query )

Triggered by birthday-cake-top-consumer!

Use formik with context/hooks instead of render-props

Note that this feature is available in formik >= 2.0

Usually when you’re creating forms using formik you access formik props using render-prop pattern

export function MyForm() {
  return (
    <Formik
      initialValues={{ /* something here */ }}
      onSubmit={(values, actions) => {/* handler here */}}
    >
      {(formikProps) => (
        <form onSubmit={formikProps.handleSubmit}>
          <MyInputComponent value={formikProps.values.name} />
        </form>
      )}
    </Formik>
  )
}

Instead you can use formik as context provider (which it is under the hood actually) and use useFormikContext() hook or FormikConsumer component to consume provided form state.

Example below:

export function MyFormContainer() {
  return (
    <Formik
      {/* Under the hood Formik uses Context Provider */}
      initialValues={{ /* something here */}}
      onSubmit={(values, actions) => { /* handler here */ }}
    >
      {/* You do not have to use function here! */}
      {/* However you can, if you would like to :) */}
      <MyForm />
    </Formik>
  )
}

export function MyForm() {
  const { handleSubmit, values } = useFormikContext(); // formikProps

  return (
    <form onSubmit={handleSubmit}>
      <MyInputComponent value={values.name} />
    </form>
  );
}

This allows us to use formik props deep down in our nested components without drilling down formikProps.

validate! can raise various classes of exceptions

class ActiveRecord::Base and module ActiveModel::Validations raise different exceptions on failed validate!

consider the following classes:

class Foo
  include ActiveModel::Validations

  validates :foo, presence: true

  def foo
    nil
  end
end

class Organisation < ActiveRecord::Base
  validates :name, presence: true
end

Take a look on classes of raised exceptions:

[2] pry(#<Api::V1::HooksController>)> Foo.new.validate!
ActiveModel::ValidationError: Validation failed: Foo can't be blank
from /Users/bartoszmaka/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/activemodel-6.0.0.rc1/lib/active_model/validations.rb:412:in `raise_validation_error'
[3] pry(#<Api::V1::HooksController>)> Organisation.new.validate!
ActiveRecord::RecordInvalid: Validation failed: Name can't be blank
from /Users/bartoszmaka/.asdf/installs/ruby/2.6.2/lib/ruby/gems/2.6.0/gems/activerecord-6.0.0.rc1/lib/active_record/validations.rb:80:in `raise_validation_error'

The highest common ancestor of those classes is StandardError

[4] pry(#<Api::V1::HooksController>)> ActiveModel::ValidationError.ancestors
=> [ActiveModel::ValidationError,
 StandardError,
 Exception,
 ActiveSupport::Dependencies::Blamable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 ActiveSupport::Tryable,
 ActiveSupport::Dependencies::Loadable,
 JSON::Ext::Generator::GeneratorMethods::Object,
 PP::ObjectMixin,
 Kernel,
 BasicObject]

[5] pry(#<Api::V1::HooksController>)> ActiveRecord::RecordInvalid.ancestors
=> [ActiveRecord::RecordInvalid,
 ActiveRecord::ActiveRecordError,
 StandardError,
 Exception,
 ActiveSupport::Dependencies::Blamable,
 ActiveSupport::ToJsonWithActiveSupportEncoder,
 Object,
 ActiveSupport::Tryable,
 ActiveSupport::Dependencies::Loadable,
 JSON::Ext::Generator::GeneratorMethods::Object,
 PP::ObjectMixin,
 Kernel,
 BasicObject]

If you’re using form objects - chances are, that you’re importing ActiveModel::Validations. Make sure that you rescue_from the proper exceptions.

Used versions: ruby 2.6.2p47, Rails 6.0.0.rc1

Decrease bundle size by importing Lodash correctly

If you use just one or a few Lodash functions in a file, try importing them directly from their respective files, one by one.

import _ from 'lodash'; // bad - 70.5K (gzipped: 24.4K)
import { has } from 'lodash'; // bad - 70.5K (gzipped: 24.4K)

import has from 'lodash/has'; // good - 9K (gzipped: 2.7K)

Alternatively, one can use babel-plugin-lodash or lodash-es. These packages come with sets of restrictions though.

It is worth noting that mixing these 2 import styles will cause the net gain to be negative. Basically, it will import the entire lodash + individual utilities twice. So, for this to be effective, the ‘good’ pattern needs to be enforced everywhere across the codebase.

Further reading

Revert^2

In the given scenario:

  1. Pull request was merged
  2. The connected branch#1 was removed
  3. Merged PR was reverted
  4. You don’t have branch#1, but you can see the reverted commit#1

All you have to do is to track the commit#1 and revert reverted commit.

$ git revert commit#1

Form with multiple submit buttons

When you need to use the form in rails with submitting to different URL, you probably will try to handle it with js. Now you don’t need to, just use formaction on submit button. The formaction override the URL in form action and used its own.

<%= form_for @article, url: {action: "create"} do |f| %>
  <%= f.text_field :title %>
  <%= f.text_area :body, size: "60x12" %>
  <%= f.submit "Create" %> #this will use URL from form
  <%= f.submit "Create Draft Article",  formaction: '/draft_articles' %> #this will use URL from formaction
<% end %>

Clean up local list of your git branches

Have you ever had to search through a long list of local branches, which are mostly obsolete for you?

To keep your local git repository slim and avoid a long list of branches while doing git branch, there is an easy strategy that can save you a bit of time.

First, always remember to push everything you did to the remote when you leave a desk, then just run in the console:

git branch | grep -v "master" | xargs git branch -D

It will remove all your local branches except master. If you need to restore any particular branch, you can fetch and checkout - it will wait for you on your remote!

Inserting data in migrations

When you encounter a problem with inserting data in migrations and your Repo is not aware about created table yet, you need to use flush()

# ...
  def up do
    create table("players") do
      add :name, :varchar, null: false
      add :color, :varchar, null: false
      add :avatar, :varchar, null: false
    end

    create index("players", [:name], unique: true)

    flush() # 👷

    Repo.insert!(%Player{...})
  end
# ...

Trigger debugger with key combination

Whenever you need to freeze your website and check its state, you can trigger debugger at any given point.

Let’s say you want to check which classes are passed when element is hovered - move your mouse over it and trigger debugger. The page will freeze and you can browse the dom tree freely.

According to Google Chrome’s DevTools documentation:

Pause script execution (if currently running) or resume (if currently paused)

You need to have your DevTools opened

MacOs: F8 or Command+\

Windows/Linux: F8 or Control+\

Using multiple commands on heroku CLI

Need to use multiple heroku CLI commands one after another and don’t want to type -a foo-bar-baz every time?

Type export HEROKU_APP=foo-bar-baz in terminal and then just happily run heroku command on foo-bar-baz application.

heroku pg:backups:capture -a foo-bar-baz

# is equivalent to:

export HEROKU_APP=foo-bar-baz
heroku pg:backups:capture

PS. After that you might want to unset HEROKU_APP to remove variable.

Replicating Rails `content_for()` in Phoenix

Typically in Rails layout you would do sth similar to this:

<%= yield :nav_actions %>

then in view (i.e. “resource_abc/show”):

<% content_for :nav_actions do %>
  <!-- whatever -->
<% end %>

To replicate this behavior in Phoenix use render_existing/3

In layout:

<%= render_existing @view_module, "nav_actions." <> @view_template, assigns %>

Then in your template folder (“resource_abc/“) you need to define extra file nav_actions.show.html.eex (for show action) - it will render content of nav_actions.show only if the file exists.

If you want to render nav_actions for all resource actions just skip @view_template:

<%= render_existing @view_module, "nav_actions",  assigns %>

In this case file should be named nav_actions.html.eex.

Babel loader transpilation for Jest

Let’s imagine a following situation: we need to create a new npm package with create-react-app, in order to bundle some files in a package to have them available in a project (it’s possible to use create-component-lib for this task). The aforementioned sometimes requires from us to update our babel config.

In my latest task I had a situation where my babel config contained invalid (for a situation) presets:

module.exports =
{
  "presets": ["@babel/preset-env", ["react-app", { "absoluteRuntime": false }]]
}

In the above example, the build process creates a transpiled code, which contains babel runtime for minimalizing package size:

var _interopRequireWildcard = require("@babel/runtime/helpers/interopRequireWildcard");

var _interopRequireDefault = require("@babel/runtime/helpers/interopRequireDefault");

Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = void 0;

var _defineProperty2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/defineProperty"));

var _objectSpread4 = _interopRequireDefault(require("@babel/runtime/helpers/esm/objectSpread"));

var _classCallCheck2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/classCallCheck"));

var _createClass2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/createClass"));

var _possibleConstructorReturn2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/possibleConstructorReturn"));

var _getPrototypeOf3 = _interopRequireDefault(require("@babel/runtime/helpers/esm/getPrototypeOf"));

var _inherits2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/inherits"));

Now when we start our test which uses a transpiled component it’s possible we receive error code like this:

Jest encountered an unexpected token

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation to specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/mobile32/projects/formik-generator/node_modules/@babel/runtime/helpers/esm/defineProperty.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){export default function _defineProperty(obj, key, value) {
                                                                                             ^^^^^^

    SyntaxError: Unexpected token export

      10 | exports.default = void 0;
      11 |
    > 12 | var _defineProperty2 = _interopRequireDefault(require("@babel/runtime/helpers/esm/defineProperty"));
         |                                               ^

      at ScriptTransformer._transformAndBuildScript (node_modules/jest-runtime/build/script_transformer.js:403:17)
      at Object.require (dist/FormGenerator.js:12:47)

This happens because by default jest ignores transformations for everything in node_modules directory.

Now we have two possibilities to resolve this problem. In the first scenario we can add transformIgnorePatterns for our jest config to transpile babel runtime module:

transformIgnorePatterns: [
  '/node_modules/(?!@babel\/runtime)'
],

In the above example transpiled code still will be smaller in application runtime but it doesn’t crash our tests.

The second option is using another babel preset for proper transpilation:

module.exports = {
  plugins: [
    '@babel/plugin-proposal-class-properties',
  ],
  presets: [
    '@babel/preset-env',
    '@babel/preset-react',
  ],
}

Now after build, we get transpiled code working in a test environment, however, our code will be bloated with helpers required to mimic transpiled features (for example class inheritance):

Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = void 0;

var _react = _interopRequireWildcard(require("react"));

var _propTypes = _interopRequireDefault(require("prop-types"));

var _formik = require("formik");

var _yup = require("yup");

var _classnames = _interopRequireDefault(require("classnames"));

var _moment = _interopRequireDefault(require("moment"));

var _BooleanField = _interopRequireDefault(require("./FieldTypes/BooleanField"));

var _EnumField = _interopRequireDefault(require("./FieldTypes/EnumField"));

var _MoneyField = _interopRequireDefault(require("./FieldTypes/MoneyField"));

var _TextField = _interopRequireDefault(require("./FieldTypes/TextField"));

var _QuarterDateField = _interopRequireDefault(require("./FieldTypes/QuarterDateField"));

Use `pluck` to fetch paginated results from S3 client

Some of AWS client calls provide responses with the limited amount of data (typically 1.000 items per response).

Example response may look as follows:

aws_client.list_objects_v2(bucket: bucket)

=> #<struct Aws::S3::Types::ListObjectsV2Output
 is_truncated=true,
 contents=
 [#<struct Aws::S3::Types::Object
    key="reports/report_2.csv",
    last_modified=2019-03-13 14:25:04 UTC,
    etag="\"5a7c05eb47dcd13a27a26d34eb13b0ec\"",
    size=466,
    storage_class="STANDARD",
    owner=nil>,
    ...
 ]
 name="awesome-bucket",
 prefix="",
 delimiter=nil,
 max_keys=1000,
 common_prefixes=[],
 encoding_type=nil,
 key_count=1000,
 continuation_token=nil,
 next_continuation_token="1wEBwtqJOGmZF5DXgu5UhTMv386wdtND0EQzkkOUEGPPeF8tC58BEbfBvfsVHKGnxNgHxvFARrcWdCPJXXgiMzUtpedrxZP2G9wu/0but8ALLHDGdZVD4OHb41DWQKocGGAOwr0wfOeN4hUoCzimKeA==",
 start_after=nil>

Because list_objects_v2 method takes continuation_token as an argument, one of the solutions to fetch all the records may be to loop through the responses using next_continuation_token until the next_continuation_token field is empty.

Instead, you can use the built-in enumerator in the response object, which will return results from all the pages (next pages will be fetched automatically by SDK):

aws_client.list_objects_v2(bucket: bucket).map { |page| page[:contents] }

=> [[#<struct Aws::S3::Types::Object
   key="reports/report_2.csv",
   last_modified=2019-03-13 14:25:04 UTC,
   etag="\"5a7c05eb47dcd13a27a26d34eb13b0ec\"",
   size=466,
   storage_class="STANDARD",
   owner=nil>,
  #<struct Aws::S3::Types::Object
   key="reports/report_1.csv",
   last_modified=2019-03-13 13:43:30 UTC,
   etag="\"dc7215c066f62c7ddedef78e123dbc7c\"",
   size=191722,
   storage_class="STANDARD",
   owner=nil>,
   ... ]

However, there is even simpler solution to achieve the same result. You can use pluck method as follows:

aws_client.list_objects_v2(bucket: bucket).pluck(:contents)

How to fix Elasticsearch 'FORBIDDEN/12/index read-only'

By default, Elasticsearch installed with homebrew on Mac OS goes into read-only mode when you have less than 5% of free disk space. If you see errors similar to this:

Elasticsearch::Transport::Transport::Errors::Forbidden:
  [403] {"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}

Or in /usr/local/var/log/elasticsearch.log you can see logs similar to:

flood stage disk watermark [95%] exceeded on [nCxquc7PTxKvs6hLkfonvg][nCxquc7][/usr/local/var/lib/elasticsearch/nodes/0] free: 15.3gb[4.1%], all indices on this node will be marked read-only

Then you can fix it by running the following commands:

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_cluster/settings -d '{ "transient": { "cluster.routing.allocation.disk.threshold_enabled": false } }'
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

Custom nginx proxy host name

server {
    listen 80 default_server;
    server_name ~^(?<developer>.+)\.dev\.selleo\.com$;
    client_max_body_size 5M;
    root   /usr/share/nginx/html;

    location / {
      resolver 8.8.8.8;
      set $backend https://$developer-secret.app.selleo.com:443;
      proxy_pass $backend;
      proxy_set_header  X-Real-IP       $remote_addr;
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

resolver does the job here!

Have you Cmd-C'd your overcommit and lost your changes?

During overcommit running I have interrupted the process and received following message

Interrupt signal received. Stopping hooks...

⚠  Hook run interrupted by user

then

Unable to restore working tree after pre-commit hooks run:
STDOUT:
STDERR:unable to refresh index

To my horror, all my changes were lost! Fortunately, those were kept in stash, so simple git stash pop helped :) More info here

FactoryBot: Constructing objects using Dry::Types

If you face error similar to the one below

Dry::Struct::Error: [YourClassName.new] :some_attribute_name is missing in Hash input

when building objects of the class using Dry::Types and FactoryBot, be advised, that

Although factory_bot is written to work with ActiveRecord out of the box, it can also work with any Ruby class. For maximum compatibility with ActiveRecord, the default initializer builds all instances by calling new on your build class without any arguments. It then calls attribute writer methods to assign all the attribute values. While that works fine for ActiveRecord, it actually doesn’t work for almost any other Ruby class.

The fix is to add following line to your factory definition

initialize_with  { new(attributes) }

Another way to use absolute paths in js 'require'

consider following files structure:

...
▾ helpers/
    timeHelper.js
    timeHelper.test.js
▾ services/
    handleSlackCommand.js
    handleSlackCommand.test.js
  index.js
  package.json
  ...

in the package.json add:

{
  ...
  "dependencies": {
    "dotenv": "^6.2.0",
    "express": "^4.16.4",
    ...
    "helpers": "file:helpers", <-- this line
  },
}

now in the services/handleSlackCommand.js I can use

const { whatever } = require('helpers/timeHelper');

instead of

const { whatever } = require('../helpers/timeHelper');

Reducing main bundle size with React lazy

On one of our projects we were able to reduce main bundle size from ~1080kB to ~820kB only by lazy loading one library (recharts).

in router file

import React, {Component, lazy, Suspense} from 'react'
//other imports

const StatisticsComponent = lazy(() => import('./path/to/component'))

export class Root extends Component {
  
  render(){
    return(
      <Router history={history}> 
        <Switch>
          // other routes
          <Suspense fallback={<div>Loading...</div>}>
            <Route exact component={StatisticsComponent} path='/statistics' />
          </Suspense>
        </Switch>
      </Router > 
      )
  }
}