JavaScript code for generating Firebase Push IDs

generate-pushid.js

/**
 * Fancy ID generator that creates 20-character string identifiers with the following properties:
 *
 * 1. They're based on timestamp so that they sort *after* any existing ids.
 * 2. They contain 72-bits of random data after the timestamp so that IDs won't collide with other clients' IDs.
 * 3. They sort *lexicographically* (so the timestamp is converted to characters that will sort properly).
 * 4. They're monotonically increasing.  Even if you generate more than one in the same timestamp, the
 *    latter ones will sort after the former ones.  We do this by using the previous random bits
 *    but "incrementing" them by 1 (only in the case of a timestamp collision).
 */
generatePushID = (function() {
  // Modeled after base64 web-safe chars, but ordered by ASCII.
  var PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';

  // Timestamp of last push, used to prevent local collisions if you push twice in one ms.
  var lastPushTime = 0;

  // We generate 72-bits of randomness which get turned into 12 characters and appended to the
  // timestamp to prevent collisions with other clients.  We store the last characters we
  // generated because in the event of a collision, we'll use those same characters except
  // "incremented" by one.
  var lastRandChars = [];

  return function() {
    var now = new Date().getTime();
    var duplicateTime = (now === lastPushTime);
    lastPushTime = now;

    var timeStampChars = new Array(8);
    for (var i = 7; i >= 0; i--) {
      timeStampChars[i] = PUSH_CHARS.charAt(now % 64);
      // NOTE: Can't use << here because javascript will convert to int and lose the upper bits.
      now = Math.floor(now / 64);
    }
    if (now !== 0) throw new Error('We should have converted the entire timestamp.');

    var id = timeStampChars.join('');

    if (!duplicateTime) {
      for (i = 0; i < 12; i++) {
        lastRandChars[i] = Math.floor(Math.random() * 64);
      }
    } else {
      // If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.
      for (i = 11; i >= 0 && lastRandChars[i] === 63; i--) {
        lastRandChars[i] = 0;
      }
      lastRandChars[i]++;
    }
    for (i = 0; i < 12; i++) {
      id += PUSH_CHARS.charAt(lastRandChars[i]);
    }
    if(id.length != 20) throw new Error('Length should be 20.');

    return id;
  };
})();

@iboxgithub This ID is for real time database, NOT firestore.

@iboxgithub This ID is for real time database, NOT firestore.

ha...is there a reason why? we use it in Firestore already :(

This key was designed before Google bought Firebase. It allows automatic sorting by time using only the key in a gigantic json file, which real time database kinda is. This was a design choice of Firebase engineers. At that time Firestore did not exist.

After Google bought Firebase, it became evident that Real Time database is too limited. Firestore was designed to be an evolution of Google's Datastore, which is completely different beast. To guarantee the (almost) endless scaling of the database, it needs evenly distributed keys, which this is not. Firestore needs truly random keys to work properly. This, to my knowledge, is because they use multiple machines behind the scenes and divide the data to different machines by the key. Now, if the key is aaaa1, aaaa2, aaaa3... all of the docs end up in single machine creating a hot spot that hit's the limits quite fast.

If you don't have much data, you won't notice it. (The data is in single machine anyway.)

thanks ;)

Line 48 is a buffer underflow when the random bytes are zzzzzzzzzzzz (either randomly, or due to the same-millisecond increment behavior). It will attempt to increment lastRandChars[-1]. In JavaScript, this will create a property on the object with key -1 and value NaN, but in other languages, it could cause far more insidious behavior.

In C, I chose to do this (rather than throw an error), but in your JS code, throwing an error seems more consistent with how other parts of it work.

// Prevent buffer underrun on overflow (when incrementing "zzzzzzzzzzzz")
// Warning: If this gets skipped, the id is no longer guaranteed to be unique,
// but most likely will be for quite awhile (it depends on where the random
// value for this millisecond started in the range).
if (i >= 0) {
    lastRandChars[i]++;
}

Itsok

web sdk source
push

repoServerTime offsets now, nextPushId is the same as generatePushId except now is passed as argument.

export function push(
  parent: DatabaseReference,
  value?: unknown
): ThenableReference {
  parent = getModularInstance(parent);
  validateWritablePath('push', parent._path);
  validateFirebaseDataArg('push', value, parent._path, true);
  const now = repoServerTime(parent._repo);
  const name = nextPushId(now);

nextPushId