You could use the jetpack_set_available_extensions
filter (reference) to avoid registering some of Jetpack's blocks in the editor. When you do so, they won't appear in the block editor.
That said, it won't change the size of the CSS file used to style those blocks in the editor; that file concatenates CSS for all blocks, regardless of what's actually available in the editor. It should not be too much of an issue though:
- That file is minified and rather small (63.8 KB right now - #).
- The file is only loaded in the editor. It is not loaded on the frontend. On your site, we only load CSS for the blocks that are on the page, and need CSS, and in those cases we use separate CSS files like the ones you can find in this directory.
I hope this clarifies things a bit.