zacharysyoung

Better way to prepend BOM

I wanted to compare solutions from JonSG and chepner to see if any ran particularly faster (particularly to see if chepner's ran faster), and to see if they only add the BOM (and don't mutate the text along the way).

Both failed, but for different reasons; JonSG's can easily be fixed.

My comparator:

1. runs and times both functions against a 10MB UTF-8 encoded file of random text that runs the full spectrum of Unicode, minus invalid UTF-16 surrogate pairs
2. reads the output and asserts the output has a BOM; also chomps the BOM leaving what should be the original UTF-8 bytes

zacharysyoung

/**
 * Make sure to check in the tab you run this script from 
 * for any kind of notification about pop-ups being blocked
 * then allow for this site/page only.
 *
 * https://stackoverflow.com/questions/63237482/open-multiple-tabs-with-javascript
 */

const anchors = document.getElementsByTagName('a');

zacharysyoung

ASCII, ISO-8859-X & Windows-1252

Character abbreviations

Abbrev	Description	Decimal	Hex
NUL	null character	0	00
SOH	start of heading	1	01
STX	start oftext	2	02

zacharysyoung

package main

import (
	"fmt"
	"slices"
	"strings"
)

// <https://codereview.stackexchange.com/questions/229042/find-neighboring-pins-on-a-numeric-keypad>
// Your colleague forgot the pin code from the door to the office.

zacharysyoung

Multiple filters for CSV

I recommend restructuring your filters from only proceeding (and indenting) if the criterium passes, to skipping the row if any criterium fails. This has a couple of benefits:

• keeping the code from creeping to the right
• you can add debug messages to print when a row doesn't match
• you can comment-out any single criterium without affecting the others

I test the participant IDs differently than you did, but your method of:

zacharysyoung

Split rows, align columns

I went for a solution that doesn't presuppose any kind of sorting: it just looks for a value and remembers in which column (on any row) it appeared.

Starting with this input:

a,b,a
c,c,b
d,e,e

zacharysyoung

Let my data be

Go's encoding/csv Reader type takes the novel (to me) approach of deciding that carriage return line feeds (CRLFs) should be replaced with newlines (LFs).

It not only replaces CRLFs that mark then end of one record and the beginning of the next—the encoding of the data—it replaces all CRLFs at the end of any line of text—the data itself.

The CSV:

ID,Data

zacharysyoung

CSV to JSON

zacharysyoung

CSV (w/TOML,YAML) to Go structs

zacharysyoung

Making it run not so slow

I mocked up a 60 MB XML by taking all the small samples in your original ZIP archive and just copying them all 200 times, which ended up with over 425k tok elements.

I then profiled your code and found a really bad culprit for chewing up time.

To process that XML took about 35 seconds:

Thu Jun 29 10:50:59 2023 profile.stats