PostgreSQL, the NoSQL Database

http://www.linuxjournal.com/content/postgresql-nosql-database

One of the most interesting trends in the computer world during the past few years has been the rapid growth of NoSQL databases. The term may be accurate, in that NoSQL databases don't use SQL in order to store and retrieve data, but that's about where the commonalities end. NoSQL databases range from key-value stores to columnar databases to document databases to graph databases.
On the face of it, nothing sounds more natural or reasonable than a NoSQL database. The "impedance mismatch" between programming languages and databases, as it often is described, means that we generally must work in two different languages, and in two different paradigms. In our programs, we think and work with objects, which we carefully construct. And then we deconstruct those objects, turning them into two-dimensional tables in our database. The idea that I can manipulate objects in my database in the same way as I can in my program is attractive at many levels.
In some ways, this is the holy grail of databases: we want something that is rock-solid reliable, scalable to the large proportions that modern Web applications require and also convenient to us as programmers. One popular solution is an ORM (object-relational mapper), which allows us to write our programs using objects. The ORM then translates those objects and method calls into the appropriate SQL, which it passes along to the database. ORMs certainly make it more convenient to work with a relational database, at least when it comes to simple queries. And to no small degree, they also improve the readability of our code, in that we can stick with our objects, without having to use a combination of languages and paradigms.
But ORMs have their problems as well, in no small part because they can shield us from the inner workings of our database. NoSQL advocates say that their databases have solved these problems, allowing them to stay within a single language. Actually, this isn't entirely true. MongoDB has its own SQL-like query language, and CouchDB uses JavaScript. But there are adapters that do similar ORM-like translations for many NoSQL databases, allowing developers to stay within a single language and paradigm when developing.
The ultimate question, however, is whether the benefits of NoSQL databases outweigh their issues. I have largely come to the conclusion that, with the exception of key-value stores, the answer is "no"—that a relational database often is going to be a better solution. And by "better", I mean that relational databases are more reliable, and even more scalable, than many of their NoSQL cousins. Sure, you might need to work hard in order to get the scaling to work correctly, but there is no magic solution. In the past few months alone, I've gained several new clients who decided to move from NoSQL solutions to relational databases, and needed help with the architecture, development or optimization.
The thing is, even the most die-hard relational database fan will admit there are times when NoSQL data stores are convenient. With the growth of JSON in Web APIs, it would be nice to be able to store the result sets in a storage type that understands that format and allows me to search and retrieve from it. And even though key-value stores, such as Redis, are powerful and fast, there are sometimes cases when I'd like to have the key-value pairs connected to data in other relations (tables) in my database.
If this describes your dilemma, I have good news for you. As I write this, PostgreSQL, an amazing database and open-source project, is set to release version 9.4. This new version, like all other PostgreSQL versions, contains a number of optimizations, improvements and usability features. But two of the most intriguing features to me are HStore and JSONB, features that actually turn PostgreSQL into a NoSQL database.
Fine, perhaps I'm exaggerating a bit here. PostgreSQL was and always will be relational and transactional, and adding these new data types hasn't changed that. But having a key-value store within PostgreSQL opens many new possibilities for developers. JSONB, a binary version of JSON storage that supports indexing and a large number of operators, turns PostgreSQL into a document database, albeit one with a few other features in it besides.
In this article, I introduce these NoSQL features that are included in PostgreSQL 9.4, which likely will be released before this issue of Linux Journal gets to you. Although not every application needs these features, they can be useful—and with this latest release of PostgreSQL, the performance also is significantly improved.

HStore

One of the most interesting new developments in PostgreSQL is that of HStore, which provides a key-value store within the PostgreSQL environment. Contrary to what I originally thought, this doesn't mean that PostgreSQL treats a particular table as a key-value store. Rather, HStore is a data type, akin to INTEGER, TEXT and XML. Thus, any column—or set of columns—within a table may be defined to be of type HSTORE. For example:

CREATE TABLE People (
    id   SERIAL,
    info HSTORE,
    PRIMARY KEY(id)
);

Once I have done that, I can ask PostgreSQL to show me the definition of the table:

\d people
              Table "public.people"

-----------------------------------------------------------------
| Column | Type    | Modifiers                                  |
-----------------------------------------------------------------
| id     | integer | not null default                           |
|        |         |  ↪nextval('people_id_seq'::regclass)|
-----------------------------------------------------------------
| info   | hstore  |                                            |
-----------------------------------------------------------------
 Indexes:
        "people_pkey" PRIMARY KEY, btree (id)

As you can see, the type of my "info" column is hstore. What I have effectively created is a (database) table of hash tables. Each row in the "people" table will have its own hash table, with any keys and values. It's typical in such a situation for every row to have the same key names, or at least some minimum number of overlapping key names, but you can, of course, use any keys and values you like.
Both the keys and the values in an HStore column are text strings. You can assign a hash table to an HStore column with the following syntax:

INSERT INTO people(info) VALUES ('foo=>1, bar=>abc, baz=>stuff');

Notice that although this example inserts three key-value pairs into the HStore column, they are stored together, converted automatically into an HStore, splitting the pairs where there is a comma, and each pair where there is a => sign.
So far, you won't see any difference between an HStore and a TEXT column, other than (perhaps) the fact that you cannot use text functions and operators on that column. For example, you cannot use the || operator, which normally concatenates text strings, on the HStore:

UPDATE People SET info = info || 'abc';
ERROR:  XX000: Unexpected end of string
LINE 1: UPDATE People SET info = info || 'abc';
                                             ^

PostgreSQL tries to apply the || operator to the HStore on the left, but cannot find a key-value pair in the string on the right, producing an error message. However, you can add a pair, which will work:

UPDATE People SET info = info || 'abc=>def';

As with all hash tables, HStore is designed for you to use the keys to retrieve the values. That is, each key exists only once in each HStore value, although values may be repeated. The only way to retrieve a value is via the key. You do this with the following syntax:

SELECT info->'bar' FROM People;
----------------
| ?column? |   |    
----------------
| abc      |   |
----------------
(1 row)

Notice several things here. First, the name of the column remains without any quotes, just as you do when you're retrieving the full contents of the column. Second, you put the name of the key after the -> arrow, which is different from the => ("hashrocket") arrow used to delineate key-value pairs within the HStore. Finally, the returned value always will be of type TEXT. This means if you say:

SELECT info->'foo' || 'a' FROM People;
----------------
| ?column? |   |
----------------
| 1a       |   |
----------------
(1 row)

Notice that ||, which works on text values, has done its job here. However, this also means that if you try to multiply your value, you will get an error message:

SELECT info->'foo' * 5 FROM People;
info->'foo' * 5 from people;
                     ^
Time: 5.041 ms

If you want to retrieve info->'foo' as an integer, you must cast that value:

SELECT (info->'foo')::integer * 5 from people;
----------------
| ?column? |   |
----------------
|   5      |   |
----------------
(1 row)

Now, why is HStore so exciting? In particular, if you're a database person who values normalization, you might be wondering why someone even would want this sort of data store, rather than a nicely normalized table or set of tables.
The answer, of course, is that there are many different uses for a database, and some of them can be more appropriate for an HStore. I never would suggest storing serious data in such a thing, but perhaps you want to keep track of user session information, without keeping it inside of a binary object.
Now, HStore is not new to PostgreSQL. The big news in version 9.4 is that GiN and GIST indexes now support HStore columns, and that they do so with great efficiency and speed.
Where do I plan to use HStore? To be honest, I'm not sure yet. I feel like this is a data type that I likely will want to use at some point, but for now, it's simply an extra useful, efficient tool that I can put in my programming toolbox. The fact that it is now extremely efficient, and its operators can take advantage of improved indexes, means that HStore is not only convenient, but speedy, as well.

JSON and JSONB

It has long been possible to store JSON inside PostgreSQL. After all, JSON is just a textual representation of JavaScript objects ("JavaScript Object Notation"), which means that they are effectively strings. But of course, when you store data in PostgreSQL, you would like a bit more than that. You want to ensure that stored data is valid, as well as use PostgreSQL's operators to retrieve and work on that data.
PostgreSQL has had a JSON data type for several years. The data type started as a simple textual representation of JSON, which would check for valid contents, but not much more than that. The 9.3 release of PostgreSQL allowed you to use a larger number of operators on your JSON columns, making it possible to retrieve particular parts of the data with relative ease.
However, the storage and retrieval of JSON data was never that efficient, and the JSON-related operators were particularly bad on this front. So yes, you could look for a particular name or value within a JSON column, but it might take a while.
That has changed with 9.4, with the introduction of the JSONB data type, which stores JSON data in binary form, such that it is both more compact and more efficient than the textual form. Moreover, the same GIN and GIST indexes that now are able to work so well with HStore data also are able to work well, and quickly, with JSONB data. So you can search for and retrieve text from JSONB documents as easily (or more) as would have been the case with a document database, such as MongoDB.
I already have started to use JSONB in some of my work. For example, one of the projects I'm working on contacts a remote server via an API. The server returns its response in JSON, containing a large number of name-value pairs, some of them nested. (I should note that using a beta version of PostgreSQL, or any other infrastructural technology, is only a good idea if you first get the client's approval, and explain the risks and benefits.)
Now, I'm a big fan of normalized data. And I'm not a huge fan of storing JSON in the database. But rather than start to guess what data I will and won't need in the future, I decided to store everything in a JSONB column for now. If and when I know precisely what I'll need, I will normalize the data to a greater degree.
Actually, that's not entirely true. I knew from the start that I would need two different values from the response I was receiving. But because I was storing the data in JSONB, I figured it would make sense for me simply to retrieve the data from the JSONB column.
Having stored the data there, I then could retrieve data from the JSON column:

SELECT id, email,
       personal_data->>'surname' AS surname
       personal_data->>'forename' as given_name
  FROM ID_Checks
 WHERE personal_data->>'surname' ilike '%lerner%';

Using the double-arrow operator (->>), I was able to retrieve the value of a JSON object by using its key. Note that if you use a single arrow (->), you'll get an object back, which is quite possibly not what you want. I've found that the text portion is really what interests me most of the time.

Conclusion

People use NoSQL databases for several reasons. One is the impedance mismatch between objects and tables. But two other common reasons are performance and convenience. It turns out that modern versions of PostgreSQL offer excellent performance, thanks to improved data types and indexes. But they also offer a great deal of convenience, letting you set, retrieve and delete JSON and key-value data easily, efficiently and naturally.
I'm not going to dismiss the entire NoSQL movement out of hand. But I will say that the next time you're thinking of using a NoSQL database, consider using one that can already fulfill all of your needs, and which you might well be using already—PostgreSQL.

Resources

Blog postings about improvements to PostgreSQL's GiN and GIST indexes, which affect the JSON and HStore types:

• http://obartunov.livejournal.com/172503.html
• http://obartunov.livejournal.com/174887.html
• http://obartunov.livejournal.com/175235.html

PostgreSQL documentation is at http://postgresql.org/docs, and it includes several sections for each of HStore and JSONB.

Crack passwords

http://www.linuxvoice.com/crack-passwords

How secure are your passwords? Find out (and stay safer online) by cracking them with John The Ripper.

Most people use passwords many times a day. They’re the keys that unlock digital doors and give us access to our computers, our email, our data and sometimes even our money. As more and more things move online, passwords secure an ever growing part of our lives. We’re told to add capital letters, numbers and punctuation to these passwords to make them more secure, but just what difference do these have? What does a really secure password look like?

In order to answer these questions, we’re going to turn into an attacker and look at the methods used to crack passwords. There are a few password-cracking tools available for Linux, but we’re going to use John The Ripper, because it’s open source and is in most distros’ repositories (usually, the package is just called john).

In order to use it, we need something to try to crack. We’ve created a file with a set of MD5-hashed passwords; they’re all real passwords that were stolen from a website and posted on the internet. MD5 is quite an old hashing method, and we’re using it because it should be relatively quick to crack on most hardware. To make matters easier, all the hashes use the same salt. Although we’ve chosen a setup that’s quick to crack, this same setup is quite common in organisations that don’t focus on security. You can download the file from here.

After downloading that file, you can try and crack the passwords with:

john md5s-short

The passwords in this file are all quite simple, and you should crack them all very quickly. Not all password hashes will surrender their secrets this easily.

When you run John The Ripper like this, it tries increasingly more complex sequences until it finds the password. If there are complex passwords, it may continue running for months or years unless you press Ctrl+C to terminate it.

Once this has finished running you can see what passwords it found with:

john --show md5s-short

That’s the simplest way of cracking passwords – and you’ve just seen that it can be quite effective – so now lets take a closer look at what just happened.

The speed at which John can crack hashes varies dramatically depending on the hashing algorithm. Slow algorithms (such as bcrypt) can be tens of thousands of times slower than quick ones like DES.

John The Ripper works by taking words from a dictionary, hashing them, and comparing these hashes with the ones you’re trying to crack. If the two hashes match, that’s the password you’re looking for. A crucial point in password cracking is how quickly you can perform these checks. You can see how fast john can run on your computer by entering:

john --test

This will benchmark a few different hashing algorithms and give their speeds in checks per second (c/s).

By default, John will run in single-threaded mode, but if you want to take full advantage of a multi-threaded approach, you can add the –fork=N option to the command where N is the number of processes. Typically, this is best where N is the number of CPU cores you want to dedicate to the task.

In the previous example, you probably found John cracked most of the passwords very quickly. This is because they were all common passwords. Since John works by checking a dictionary of words, common passwords are very easy to find.

John comes with a word list that it uses by default. This is quite good, but to crack more and more secure passwords, you then need a word list with more words. People who crack passwords regularly often build their own word lists over years, and they can come from many sources. General dictionaries are good places to start (which languages you pick will depend on your target demographic), but these don’t usually contain names, slang or other terms.

Crackers regularly steal passwords from organisations (often websites) and post them online. These password leaks may contain thousands or even millions of passwords, so these are a great source of extra words. Good word lists are often sold (such as https://crackstation.net/buy-crackstation-wordlist-password-cracking-dictionary.htm, which is pay-what-you-want). This latter has about 1.5 billion words; even larger word lists are available, but usually for a fee.

With John, you can use a custom word list with the –wordlist= option. For example, to check passwords using your system’s dictionary, use:

rm ~/.john/john.pot
john --wordlist=/usr/share/dict/words md5s-short

This should work on most Debian-based systems, but on other distros, the words file may be in a different place. The first line deletes the file that contains the cracked passwords. If you don’t run this, it won’t bother trying to crack anything, as it already has all the passwords. The regular dictionary isn’t as good as John The Ripper’s dictionary, so this won’t get all the passwords.

How passwords work

Passwords present something of a computing conundrum. When people enter their password, the computer has to be able to check that they’ve entered the right password. At the same time though, it’s a bad idea to store passwords anywhere on the computer, since that would mean that any hacker or malware might be able to get the passwords file and then compromise every user account.

Hashing (AKA one-way encryption) is the solution to this problem. Hashing is a mathematical process that scrambles the password so that it’s impossible to unscramble it (hence one-way encryption).

When you set the password, the computer hashes it and stores the hash (but not the password). When you enter the password, the computer then hashes it and compares this hash to the stored hash. If they’re the same, then the computer assumes that the passwords are the same and therefore lets you log in.

There are a few things make a good hashing algorithm. Obviously, it should be impossible to reverse (otherwise it’s not a hashing algorithm), but other than this, it should minimise the number of collisions. This is where two different things produce the same hash, and the computer would therefore accept both as valid. It was a collision in the MD5 hashing algorithm that allowed the Flame malware to infiltrate the Iranian Oil Ministry and many other government organisations in the Middle East.

Another important thing about good hashing algorithms is that they’re slow. That might sound a little odd, since generally algorithms are designed to be fast, but the slower a hash is, the harder it is to crack. For normal use, it doesn’t make much difference if the hash takes 0.000001 seconds or 0.001 seconds, but the latter takes 1,000 times longer to crack.

You can get a reasonable idea of how fast or slow an algorithm is by running john –test to benchmark the different algorithms on your computer. The fewer checks per second, the slower it will be for an attacker to break any hashes using that algorithm.

Mangling words

Secure services often place rules on what passwords are allowed. For example, they might insist on upper and lower case letters as well as numbers or punctuation. In general, people won’t add these randomly, but put them in words in specific ways. For example, they might add a number to the end of a word, or replace letters in a word with punctuation that looks similar (such as a with @).

John The Ripper provides the tools to mangle words in this way, so that we can check these combinations from a normal word list.

For this example, we’ll use the password file from www.linuxvoice.com/passwords, which contains the passwords: password, Password, PASSWORD, password1, p@ssword, P@ssword, Pa55w0rd, p@55w0rd. First, create a new text file called passwordlist containing just:

password

This will be the dictionary, and we’ll create rules that crack all the passwords based of this one root word.

Rules are specified in the john.conf file. By default, john uses the configuration files in ~/.john, so you’ll need to create that file in a text editor. We’ll start by adding the lines:

[List.Rules:Wordlist]
:
c

The first line tells john what mode you want to use the rules for, end every line below that is a rule (we’ll add more in a minute). The : just tells John to try the word as it is, no alterations, while c stands for capitalise, which makes the first character of the word upper case. You can try this out with:

john passwords.md5 --wordlist=passwordlist --rules

You should now crack two of the passwords despite there only being one word in the dictionary. Let’s try and get a few more now. Add the following to the config file:

u
$[0-9]

The first line here makes the whole word upper case. On the second line, the $ symbol means append the following character to the password. In this case, it’s not a single character, but a class of characters (digits), so it tries ten different words (password0, password1… password9).

To get the remaining passwords, you need to add the following rules to the config file:

csa@
sa@so0ss5
css5so0

The rule s replaces all occurrences of character1 with character2. In the above rules, this is used to switch a for @ (sa@), o for 0 (so0) and s for 5 (ss5). All of these are combination rules that build up the final word through more than one alteration.

Processing power

The faster your computer can hash passwords, the more you can try in a given amount of time, and therefore the better chance you have of cracking the password. In this article, we’ve used John The Ripper because it’s an open source tool that’s available on almost all Linux platforms. However, it’s not always the best option. John runs on the CPU, but password hashing can be run really efficiently on graphics cards.

Hashcat is password cracking program that runs on graphics cards, and on the right hardware can perform much better than John. Specialised password cracking computers usually have several high-performance GPUs and rely on these for their speed.

You probably won’t find Hashcat in your distro’s repositories, but you can download it from www.hashcat.net (it’s free as in zero cost, but not free as in free software). It comes in two flavours: ocl-Hashcat for OpenCL cards (AMD), and cuda-Hashcat for Nvidia cards.

Raw performance, of course, means very little without finesse, so fancy hardware with GPU crackers means very little if you don’t have a good set of words and rules.

A text-menu driven tool for creating John The Ripper config files is available from this page.

Limitations of cracking rules

The language for creating rules isn’t very expressive. For example, you can’t say: ‘try every combination of the following rules’. The reason for that is speed. The rules engine has to be able to run thousands or even millions of times per second while not significantly slowing down the hashing.

You’ve probably guessed by now that creating a good set of rules is quite a time-consuming process. It involves a detailed knowledge of what patterns are commonly used to create passwords, and an understanding of the archaic syntax used in the rules engines. It’s good to have an understanding of how they work, but unless you’re a professional penetration tester, it’s usually best to use a pre-created rule list.

The default rules with John are quite good, but there are some more complex ones available. One of the best public ones comes from a DefCon contest in 2010. You can grab the ruleset from the website: http://contest-2010.korelogic.com/rules.html.

You’ll get a file called rules.txt, which is a John The Ripper configuration file, and there are some usage examples on the above website. However, it’s not designed to work with the default version of John The Ripper, but a patched version (sometimes called -jumbo). This isn’t usually available in distro repositories, but it can be worth compiling it because it has more features than the default build. To get it, you’ll need to clone it from GitHub with:

git clone https://github.com/magnumripper/JohnTheRipper
cd JohnTheRipper/

There are a few options in the install procedure, and these are documented in JohnTheRipper/doc/Install. We compiled it on an Ubuntu 14.04 system with:

cd JohnTheRipper/src
./configure && make -s clean && make -sj4

This will leave the binary JohnTheRipper/run/john that you can execute. It will expect the john.conf file (which can be the file downloaded from KoreLogic) in the same directory.

If you don’t want to compile the -jumbo version of John, you can still use the rules from KoreLogic, you’ll just have to integrate them into a john.conf file by hand first. There are a lot of rules, so you’ll probably want to pick out a few, and copy them into the john.conf file in the same way you did when creating the rules earlier, and omit the lines with square brackets.

As you’ve seen, cracking passwords is part art and part science. Although it’s often thought of as a malicious practice, there are some real positive benefits of it. For example, if you run an organisation, you can use cracking tools like John to audit the passwords people have chosen. If they can be cracked, then it’s time to talk to people about computer security. Some companies run periodic checks and offer a small reward for any employee whose password isn’t cracked. Obviously, all of these should be done with appropriate authorisation, and you should never use a password cracker to attack someone else’s password except when you have explicit permission.

John The Ripper is an incredibly powerful tool whose functionality we’ve only just touched on. Unfortunately, its more powerful features (such as its rule engine) aren’t well documented. If you’re interested in learning more about it, the best way of doing this is by generating hashes and seeing how to crack them. It’s easy to generate hashes by simply creating new users in your Linux system and giving them a password; then you can copy the /etc/shadow file to your home directory and change the owner with:

sudo cp /etc/shadow ~
sudo chown  ~/shadow

Where is your username. You can then run John on the shadow file. If you’ve got a friend who’s interested in cracking as well, you could create challenges for each other (remember to delete the lines for real users from the shadow file though!). Alternatively, you can try our shadow file for the latest in our illustrious series of competitions.

So, what does a secure password look like? Well, it shouldn’t be based on a dictionary word. As you’ve seen, word mangling rules can find these even if you’ve obscured it with numbers or punctuation. It should also be long enough to make brute force attacks impossible (at least 10 characters). Beyond that, it’s best to use your own method, because any method that becomes popular can be exploited by attackers to create better word lists and rules.

Build your own combined OpenVPN/WiKID server for a VPN with built-in two-factor authentication using Packer

https://www.howtoforge.com/tutorial/build-wikid-openvpn-server-with-packer

In past tutorials, we have added one-time passwords to OpenVPN and created a WiKID server using Packer. In this tutorial we create a combined OpenVPN/WiKID server using Packer. Packer allows us to create VMware, VirtualBox, EC2, GCE, Docker, etc images using code. Note that combining your two-factor authentication server and VPN server on one box may or may not be the best solution for you. We typically like separation of duties for security and flexibility. However, if you need something fast - the PCI auditors arrive Monday - or you are in a repressive state and just need a secure outbound connection for a short period of time. And you still have some flexibility. You can add more services to the WiKID server. You can disable the OpenVPN server and instead switch to a different VPN.

Build the Combined Server

First, download and install Packer.
Checkout our Packer scripts from GitHub. The scripts consist of a main JSON file that tells Packer what do it, an http directory with Anaconda build scripts, a files directory that gets uploaded to the image and provisioners that run after the image is built. Basically Packer starts with some source such as an ISO or AMI, builds the server based on Anaconda (at least for CentOS), uploads any files and then runs the provisioners. Packer is primarily geared toward creating idempotent servers. In our case, we are using it to execute the commands, allowing us to run one command instead of about 50 (just for the provisioning).
Before building, you need to edit a few files. First, edit /files/vars. This is the standard vars file for creating the OpenVPN certs. Just enter your values for the cert fields.

# These are the default values for fields
# which will be placed in the certificate.
# Don't leave any of these fields blank.
export KEY_COUNTRY="US"
export KEY_PROVINCE="GA"
export KEY_CITY="Atlanta"
export KEY_ORG="WiKID Systems Inc"
export KEY_EMAIL="me@wikidsystems.com"
export KEY_OU="WiKID Systems, Inc"

Next, you need to edit the shared secret in /files/server. This file will tell PAM which RADIUS server to use. In this case, it is talking straight to the WiKID server. The shared secret is used to encode the radius traffic. Since WiKID is running on the same server, keep the localhost as the server:

# server[:port] shared_secret      timeout (s)
127.0.0.1       secret             3
 

You will need this shared secret later.
Take a look in centos-6-x86-64.json. You can run it as is, but you might like to edit a few things. You should confirm the source_ami (the listed ami is in the US-East) or switch it to one of your preferred CentOS AMIs. If you are building on VMware or VirtualBox, you will want to change the iso_url to the location of the CentOS ISO on your hard drive and update the MD5Sum. You may want to edit the names and descriptions. You may also want to change the EC2 region. Most importantly, you can change the ssh_password which is the root password.
Once you are happy with the JSON file, you can verify it with Packer:

$packer_location/packer verify centos-6-x86-64.json

If that works, build it. You can specify the target platform on the command line:

$packer_location/packer build --only=virtualbox-iso centos-6-x86-64.json

If you build for EC2, put the required credentials in the command line:

$packer_location/packer build -var 'aws_access_key=XXXXXXXXXXXXXXXXXXXX' -var 'aws_secret_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' --only=amazon-ebs centos66.json

If you watch the commands run, you will see a complete OpenVPN server being built complete with fresh certificates!

Configure the WiKID two-factor authentication server

Once it is created, you will need to launch the AMI or import the virtual machine. Start VirtBox and select File, Import Appliance. Point it to output-virtualbox-iso directory created by the build command and open the OVF file. Make any changes you may want to the virtual machine (eg memory or network) and start the server.
Login using root/wikid or whatever you may have set the root password as in the JSON file. We will be configuring the WiKID server using the quick-start configuration option. Copy the file to the current directory:

cp /opt/WiKID/conf/sample-quick-setup.properties wikid.conf

Edit wikid.conf per those instructions. Use the external IP address of your server or EC2 instance zero-padded as the domaincode. So, 54.163.165.73 becomes 054163165073. For the RADIUS host use the localhost and the shared secret you created in /files/server above:

information for setting up a RADIUS host
radiushostip=127.0.0.1
radiushostsecret=secret
; *NOTE*: YOU SHOULD REMOVE THIS SETTING AFTER CONFIGURATION FOR SECURITY
  

If you are on a VM, you can configure the network by running:

wikidctl setup

On EC2, you can just configure the WiKID server:

wikidctl quick-setup configfile=wikid.conf

You will see configuration information scroll past. Start the WiKID server:

wikidctl start

You will be prompted for the passphrase you set in wikid.conf. Browse to the WIKIDAdmin interface at https://yourserver.com/WiKIDAdmin/ and you should see your domain created, your radius network client configured and all the required certs completed.
Before leaving the server, you should add your username as an account on the server with 'useradd $username'. There is no need to add a password.

Register the WiKID Software token

Download a WiKID software token or install one for iOS or Android from the apps stores.
Start the token and select "Add a Domain". Enter the domain identifier code you set in wikid.conf and you should be double-prompted to set your PIN. Do so and you will get back a registration code. Go to the WiKIDAdmin web UI and click on the Users Tab, then Manually Validate a User. Click on your registration code and enter your username. This process associates the token (and the keys that were exchanged) with the user.

Setup the VPN client

Download the ca.crt to the client:

scp -i ~/Downloads/wikid.pem root@yourserver.com:/etc/openvpn/ca.crt .

Edit the client.conf OpenVPN file. Set the remote server as your combined WiKID/OpenVPN server:

# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
remote yourserver.com 1194
   

Comment out the lines for the cert and key. Leaving only the CA. Since we are using WiKID to authenticate and identify the user, they are not needed.

ca ca.crt
#cert client.crt
#key client.key
 

At the bottom of the file, tell the client to prompt for a password:

auth-user-pass

Now start the OpenVPN client:

sudo openvpn client.conf

You will be prompted for a username and a password. Request a passcode from your WiKID token and enter it into the password field. You should be granted accesss.
Related:

• WiKID - Two-factor authentication Server
• OpenVPN
• Packer