rwest/README

## README
These two files should help you to import passwords from mac OS X keychains to 1password.

Assumptions:
1) You have some experience with scripting/are a power-user. These scripts worked for me
   but they haven't been extensively tested and if they don't work, you're on your own!
   Please read this whole document before starting this process. If any of it seems
   incomprehensible/frightening/over your head please do not use these scripts. You will
   probably do something Very Bad and I wouldn't want that.
2) You have ruby 1.9.2 installed on your machine. This comes as standard with Lion, previous
   versions of OS X may have earlier versions of ruby, which *may* work, but then again, they
   may not :-) You can check by opening the terminal application and typing ruby -v.
   NB. The script has now been modified to work with ruby 1.8.7 (which is actually standard with Lion.
3) *THIS IS IMPORTANT* None of your passwords, usernames or site names contains a comma. It's
   highly unlikely that a site name will contain a comma, fairly unlikely that usernames will,
   but eminently possible that your passwords might. If they do, this script *will not work*
   as supplied. You can modify it to quote all the values (there's a function for this already
   in the script) before it outputs them, but beware: if any of your passwords contains a "
   character it will break if you do this. If you have both quotes and commas in your passwords,
   well, damn, you're fresh out of luck. The best you can do is to find the passwords with commas
   in and remove them manually from the exported keychain (I'll mention where to do this below)

Instructions:

0) Save keychain.rb and click_allow.scpt in your home directory.

1) Enable full GUI scripting by going to the Universal Access System Preference Pane
   and checking "Enable access for assistive devices"

2) Open the Terminal application and run the following command:
   security dump-keychain -d login.keychain > keychain.txt

(If you have multiple keychains you should repeat this whole process once from step 2 onwards for
 each one. You will have to change 'login.keychain' to 'foo.keychain' or somesuch.)

3) When you run the above command, the system will ask for permission to use your keychain. If you
   have a separate keychain password/have paranoid settings on your keychain, you may need to enter
   a password now. Otherwise, you will be presented with a dialog box asking you whether you want to
   allow permission to access the first item in your keychain. You will be asked this once for every
   item in your keychain (zzz). This is where the other file comes in:

4) Find the click_allow.scpt in your home directory using Finder, double click it. It will open in
   the AppleScript editor. Click the run button. If all is well, the script will click the "Allow"
   button for you lots of times until all of your keychain entries have been exported. Shouldn't
   take more than a few minutes even for hundreds of entries.

5) When that finishes, go back to the Terminal window and run the following command:
   ruby keychain.rb keychain.txt | sort > keychain.csv

6) If all is well, that command will finish very quickly without any message. If it spouts an error
   at you, sorry, you'll have to fix the script, something's broken. Otherwise you should try opening
   up keychain.csv in your favourite text editor (TextEdit? <shiver>) to make sure it contains a list
   of keychain entries. Now is the time to search for passwords containing a comma (you may need regular
   expressions to do this if you have a lot of keychain entries, since it's a comma-separated file)
   and delete them to stop them hosing the 1password import. You'll have to enter these manually, hopefully
   it isn't too many.

7) Fire up 1password and choose File>Import. You want to import keychain.csv as a "CSV or Delimited Text"
   file. The process is fairly self-explanatory, make sure you select "comma" as the delimiter at the
   appropriate point. You will have to tell it which columns correspond to which fields (this is pretty
   obvious) and you should check that there are exactly five columns. If you're seeing more than five
   columns, one of your values contains a rogue comma and you need to fix it manually before you import the
   file or it won't work. The 5th column is optional - it's the last modified date for the keychain entry;
   unfortunately 1password won't let you import this as the "modified date" for the password but I put
   it in a notes field just in case since I often find it helpful to know when a password was set.

8) IMPORTANT: You now have 2 files on your hard disk that contain unencrypted passwords. You need to delete
   these securely if you are concerned about the possibility that someone might get your passwords. You have
   two options. The easy option is to use Finder to move them to Trash, and then Secure Empty Trash. If you
   are one of these funny people who likes to use their Trash Can as a temporary storage location and don't
   want to empty it, you can go back to the terminal and issue rm  keychain.csv keychain.txt, and then fire up Disk
   Utility and use the "Erase Free Space" command on the relevant hard disk to securely blank all the free
   space on your drive (this may take some time). NB: If you have an SSD drive in your computer there will be
   no Secure Empty Trash (only plain Empty Trash) and there will be no "Erase Free Space" in Disk Utility.
   This is because some SSDs delete things much more permanently than traditional hard disks by default, so
   these commands are redundant. Simply emptying the trash/rm-ing the file from the terminal will suffice in
   this case.

Acknowledgements: The original ruby script was written by Morgan Schweers of https://github.com/cyberfox. I've merely fixed bits that didn't work for me, and added the script to push the Allow button + this documentation.

## click_allow.scpt
tell application "System Events"
	repeat while exists (processes where name is "SecurityAgent")
		tell process "SecurityAgent"
			click button "Allow" of group 1 of window 1
		end tell
		delay 0.2
	end repeat
end tell

## keychain.rb
#!/usr/bin/env ruby
#
# Usage:
#   security dump-keychain -d login.keychain > keychain_logins.txt
#   # Lots of clicking 'Always Allow', or just 'Allow', until it's done...
#   curl -O curl -O https://raw.github.com/gist/1224792/06fff24412311714ad6534ab700a7d603c0a56c9/keychain.rb
#   chmod a+x ./keychain.rb
#   ./keychain.rb keychain_logins.txt | sort > logins.csv
#
# Then import logins.csv in 1Password using the format:
# Title, URL/Location, Username, Password
# Remember to check 'Fields are quoted', and the Delimiter character of 'Comma'.
require 'date'

class KeychainEntry
  attr_accessor :fields

  def initialize(keychain)
    last_key = nil
    @fields = {}
    data = nil
    aggregate = nil
    lines = keychain.split("\n")
    lines.each do |line|
      # Everything after the 'data:' statement is data.

      if data != nil
        data << line
      elsif aggregate != nil
        if ( line[0] == 32 || line[0] == " " )
          keyvalue = line.split('=', 2).collect { |kv| kv.strip }
          aggregate[keyvalue.first] = keyvalue.last
        else
          @fields[last_key] = aggregate
          aggregate = nil
        end
      end

      if aggregate == nil
        parts = line.split(':').collect { |piece| piece.strip }
        if parts.length > 1
          @fields[parts.first] = parts.last
        else
          last_key = parts.first
          data = [] if parts.first == "data"
          aggregate = {}
        end
      end
    end
    @fields["data"] = data.join(" ") if data
  end
end

def q(string)
  "\"#{string}\""
end

def process_entry(entry_string)
  entry = KeychainEntry.new(entry_string)


  if entry.fields['class'] == '"inet"' && ['"form"', '"dflt"'].include?(entry.fields['attributes']['"atyp"<blob>'])
    site = entry.fields['attributes']['"srvr"<blob>'].gsub!('"', '')
    path = entry.fields['attributes']['"path"<blob>'].gsub!('"', '')
    proto= entry.fields['attributes']['"ptcl"<uint32>'].gsub!('"', '')
    proto.gsub!('htps', 'https');
    user = entry.fields['attributes']['"acct"<blob>'].gsub!('"', '')
    #user = entry.fields['attributes']['0x00000007 <blob>'].gsub!('"', '')
    date_string = entry.fields['attributes']['"mdat"<timedate>'].gsub(/0x[^ ]+[ ]+/, '').gsub!('"', '')
    date = DateTime.parse(date_string)
    pass = entry.fields['data'][1..-2]
    path = '' if path == '<NULL>'
    url = "#{proto}://#{site}#{path}"

    puts "#{site},#{url},#{user},#{pass},#{date}"
    #puts "#{user}, #{pass}, #{date}"
  end
end

accum = ''
ARGF.each_line do |line|
  if line =~ /^keychain: /
    unless accum.empty?
      process_entry(accum)
      accum = ''
    end
  end
  accum += line
end
	These two files should help you to import passwords from mac OS X keychains to 1password.

	Assumptions:
	1) You have some experience with scripting/are a power-user. These scripts worked for me
	but they haven't been extensively tested and if they don't work, you're on your own!
	Please read this whole document before starting this process. If any of it seems
	incomprehensible/frightening/over your head please do not use these scripts. You will
	probably do something Very Bad and I wouldn't want that.
	2) You have ruby 1.9.2 installed on your machine. This comes as standard with Lion, previous
	versions of OS X may have earlier versions of ruby, which may work, but then again, they
	may not :-) You can check by opening the terminal application and typing ruby -v.
	NB. The script has now been modified to work with ruby 1.8.7 (which is actually standard with Lion.
	3) THIS IS IMPORTANT None of your passwords, usernames or site names contains a comma. It's
	highly unlikely that a site name will contain a comma, fairly unlikely that usernames will,
	but eminently possible that your passwords might. If they do, this script will not work
	as supplied. You can modify it to quote all the values (there's a function for this already
	in the script) before it outputs them, but beware: if any of your passwords contains a "
	character it will break if you do this. If you have both quotes and commas in your passwords,
	well, damn, you're fresh out of luck. The best you can do is to find the passwords with commas
	in and remove them manually from the exported keychain (I'll mention where to do this below)

	Instructions:

	0) Save keychain.rb and click_allow.scpt in your home directory.

	1) Enable full GUI scripting by going to the Universal Access System Preference Pane
	and checking "Enable access for assistive devices"

	2) Open the Terminal application and run the following command:
	security dump-keychain -d login.keychain > keychain.txt

	(If you have multiple keychains you should repeat this whole process once from step 2 onwards for
	each one. You will have to change 'login.keychain' to 'foo.keychain' or somesuch.)

	3) When you run the above command, the system will ask for permission to use your keychain. If you
	have a separate keychain password/have paranoid settings on your keychain, you may need to enter
	a password now. Otherwise, you will be presented with a dialog box asking you whether you want to
	allow permission to access the first item in your keychain. You will be asked this once for every
	item in your keychain (zzz). This is where the other file comes in:

	4) Find the click_allow.scpt in your home directory using Finder, double click it. It will open in
	the AppleScript editor. Click the run button. If all is well, the script will click the "Allow"
	button for you lots of times until all of your keychain entries have been exported. Shouldn't
	take more than a few minutes even for hundreds of entries.

	5) When that finishes, go back to the Terminal window and run the following command:
	ruby keychain.rb keychain.txt \| sort > keychain.csv

	6) If all is well, that command will finish very quickly without any message. If it spouts an error
	at you, sorry, you'll have to fix the script, something's broken. Otherwise you should try opening
	up keychain.csv in your favourite text editor (TextEdit? <shiver>) to make sure it contains a list
	of keychain entries. Now is the time to search for passwords containing a comma (you may need regular
	expressions to do this if you have a lot of keychain entries, since it's a comma-separated file)
	and delete them to stop them hosing the 1password import. You'll have to enter these manually, hopefully
	it isn't too many.

	7) Fire up 1password and choose File>Import. You want to import keychain.csv as a "CSV or Delimited Text"
	file. The process is fairly self-explanatory, make sure you select "comma" as the delimiter at the
	appropriate point. You will have to tell it which columns correspond to which fields (this is pretty
	obvious) and you should check that there are exactly five columns. If you're seeing more than five
	columns, one of your values contains a rogue comma and you need to fix it manually before you import the
	file or it won't work. The 5th column is optional - it's the last modified date for the keychain entry;
	unfortunately 1password won't let you import this as the "modified date" for the password but I put
	it in a notes field just in case since I often find it helpful to know when a password was set.

	8) IMPORTANT: You now have 2 files on your hard disk that contain unencrypted passwords. You need to delete
	these securely if you are concerned about the possibility that someone might get your passwords. You have
	two options. The easy option is to use Finder to move them to Trash, and then Secure Empty Trash. If you
	are one of these funny people who likes to use their Trash Can as a temporary storage location and don't
	want to empty it, you can go back to the terminal and issue rm keychain.csv keychain.txt, and then fire up Disk
	Utility and use the "Erase Free Space" command on the relevant hard disk to securely blank all the free
	space on your drive (this may take some time). NB: If you have an SSD drive in your computer there will be
	no Secure Empty Trash (only plain Empty Trash) and there will be no "Erase Free Space" in Disk Utility.
	This is because some SSDs delete things much more permanently than traditional hard disks by default, so
	these commands are redundant. Simply emptying the trash/rm-ing the file from the terminal will suffice in
	this case.

	Acknowledgements: The original ruby script was written by Morgan Schweers of https://github.com/cyberfox. I've merely fixed bits that didn't work for me, and added the script to push the Allow button + this documentation.
	tell application "System Events"
	repeat while exists (processes where name is "SecurityAgent")
	tell process "SecurityAgent"
	click button "Allow" of group 1 of window 1
	end tell
	delay 0.2
	end repeat
	end tell
	#!/usr/bin/env ruby
	#
	# Usage:
	# security dump-keychain -d login.keychain > keychain_logins.txt
	# # Lots of clicking 'Always Allow', or just 'Allow', until it's done...
	# curl -O curl -O https://raw.github.com/gist/1224792/06fff24412311714ad6534ab700a7d603c0a56c9/keychain.rb
	# chmod a+x ./keychain.rb
	# ./keychain.rb keychain_logins.txt \| sort > logins.csv
	#
	# Then import logins.csv in 1Password using the format:
	# Title, URL/Location, Username, Password
	# Remember to check 'Fields are quoted', and the Delimiter character of 'Comma'.
	require 'date'

	class KeychainEntry
	attr_accessor :fields

	def initialize(keychain)
	last_key = nil
	@fields = {}
	data = nil
	aggregate = nil
	lines = keychain.split("\n")
	lines.each do \|line\|
	# Everything after the 'data:' statement is data.

	if data != nil
	data << line
	elsif aggregate != nil
	if ( line[0] == 32 \|\| line[0] == " " )
	keyvalue = line.split('=', 2).collect { \|kv\| kv.strip }
	aggregate[keyvalue.first] = keyvalue.last
	else
	@fields[last_key] = aggregate
	aggregate = nil
	end
	end

	if aggregate == nil
	parts = line.split(':').collect { \|piece\| piece.strip }
	if parts.length > 1
	@fields[parts.first] = parts.last
	else
	last_key = parts.first
	data = [] if parts.first == "data"
	aggregate = {}
	end
	end
	end
	@fields["data"] = data.join(" ") if data
	end
	end

	def q(string)
	"\"#{string}\""
	end

	def process_entry(entry_string)
	entry = KeychainEntry.new(entry_string)


	if entry.fields['class'] == '"inet"' && ['"form"', '"dflt"'].include?(entry.fields['attributes']['"atyp"<blob>'])
	site = entry.fields['attributes']['"srvr"<blob>'].gsub!('"', '')
	path = entry.fields['attributes']['"path"<blob>'].gsub!('"', '')
	proto= entry.fields['attributes']['"ptcl"<uint32>'].gsub!('"', '')
	proto.gsub!('htps', 'https');
	user = entry.fields['attributes']['"acct"<blob>'].gsub!('"', '')
	#user = entry.fields['attributes']['0x00000007 <blob>'].gsub!('"', '')
	date_string = entry.fields['attributes']['"mdat"<timedate>'].gsub(/0x[^ ]+[ ]+/, '').gsub!('"', '')
	date = DateTime.parse(date_string)
	pass = entry.fields['data'][1..-2]
	path = '' if path == '<NULL>'
	url = "#{proto}://#{site}#{path}"

	puts "#{site},#{url},#{user},#{pass},#{date}"
	#puts "#{user}, #{pass}, #{date}"
	end
	end

	accum = ''
	ARGF.each_line do \|line\|
	if line =~ /^keychain: /
	unless accum.empty?
	process_entry(accum)
	accum = ''
	end
	end
	accum += line
	end