Saturday, April 2, 2016

Advantages disadvantages of using Proxy vs VPN vs TOR vs TOR and VPN together

http://www.blackmoreops.com/2016/01/04/advantages-disadvantages-of-using-proxy-vs-vpn-vs-tor

What makes you truly anonymous and leaves next to no trace in the internet? In this post I will discuss the advantages disadvantages of Proxy vs VPN vs TOR vs TOR and VPN together.
Ultimate anonymity Advantages and disadvantages of using Proxy, VPN, TOR and TOR and VPN together - blackMORE Ops - 6Internet is a huge and gravely important part of our life these days. It is not anymore so simple “I will find your IP address (and DoS you)” and thus anonymity has became more complex as surveillance became more probing than ever. As more and more things are connected to the ‘Web’, you can potentially expose more info, exploit more, protect more or plunder more than ever. It is very important to be able to monitor and surveillance effectively and at the same time it is very important to be able to bypass surveillance and  monitoring. Why, that’s an entirely different philosophical discussion and I will simply leave that to the users imagination. Understanding Proxy vs VPN vs TOR vs TOR and VPN together is important as these are main methods used for anonymity.
If you are wondering how you can browse the web anonymously, you will be happy to know that there are numerous different methods guaranteeing that nobody is ever going to track you down and threaten you digitally. However, not all of these methods are equally effective and qualitative and this is what has led us in this article.
In this article, I will discuss the pros and cons of using methods like:
  1. Proxy,
  2. VPN
  3. TOR and
  4. TOR and VPN together.
This article will benefit you greatly if you are a mafia Whistle-blower, or a dissident living under regime in which it would be very dangerous to be caught performing certain activity (such as writing political blogs) and such.

What is a Proxy?

A proxy directs your traffic via another computer instead of your own. While there are public and private proxy servers, only private proxies, usually paid, provide any type of stability or reliability. Ultimate anonymity - blackMORE Ops - 3

SOCKS vs HTTP Proxy

A SOCKS server is a general purpose proxy server that establishes a TCP connection to another server on behalf of a client, then routes all the traffic back and forth between the client and the server. It works for any kind of network protocol on any port. SOCKS Version 5 adds additional support for security and UDP. The SOCKS server does not interpret the network traffic between client and server in any way, and is often used because clients are behind a firewall and are not permitted to establish TCP connections to servers outside the firewall unless they do it through the SOCKS server. Most web browsers for example can be configured to talk to a web server via a SOCKS server. Because the client must first make a connection to the SOCKS server and tell it the host it wants to connect to, the client must be “SOCKS enabled.” On Windows, it is possible to “shim” the TCP stack so that all client software is SOCKS enabled. A free SOCKS shim is available from Hummingbird at http://www.hummingbird.com/products/nc/socks/index.html.
An HTTP proxy is similar, and may be used for the same purpose when clients are behind a firewall and are prevented from making outgoing TCP connections to servers outside the firewall. However, unlike the SOCKS server, an HTTP proxy does understand and interpret the network traffic that passes between the client and downstream server, namely the HTTP protocol. Because of this the HTTP proxy can ONLY be used to handle HTTP traffic, but it can be very smart about how it does it. In particular, it can recognize often repeated requests and cache the replies to improve performance. Many ISPs use HTTP proxies regardless of how the browser is configured because they simply route all traffic on port 80 through the proxy server.

Proxy advantages

  1. Proxies are like a Web Filter. The proxy settings are applied to your internet browser whether you’re using MSIE, Chrome, Safari, Firefox, etc.
  2. When browsing the internet through a proxy server, all benefits with that server are used, for example, (Security, Speed, and Geographic Location).
  3. The proxy will only secure traffic via the internet browser using the proxy server settings.

Proxy disadvantages

  1. The primary proxy protocols in use today are SOCKS and HTTP/HTTPS. SOCKS and HTTP proxies provide no encryption, whereas HTTPS proxies offer the same level of encryption as any SSL website.
  2. Proxies were not designed to protect all of your Internet traffic, usually only the browser.
  3. Many proxies pass the user’s original IP address to the destination site, making them unsuitable for security or privacy conscious users.
  4. Proxies must be configured separately for each application (email, browser, third-party apps) and some applications may not support them.

What is a VPN?

Ultimate anonymity - blackMORE Ops - 5A Virtual Private Network is a network connection that enables you to create a secure connection to another location, thereby allowing you to appear as if you were in another place. Your computer creates an encrypted virtual tunnel to the VPN server and all of your browsing appears as if it is coming from the VPN server. All of the Internet traffic goes through this encrypted tunnel, keeping your data from being exposed to eavesdroppers between your computer and the VPN server.
Unlike a Proxy, a VPN service provider(Virtual Private Network) encrypts all of your traffic, replacing your ISP and routing ALL traffic through the VPN server, including all programs and applications. This also will use all benefits of the VPN server such as (Speed, Geopgraphic Location, and Security).
The VPN can be easily connected or disconnected with the click of a button.

VPN advantages

  1. Fast – generally speaking you will see very little slowdown to your raw internet connection speeds when using a VPN service
  2. Location spoofing is very easy – most VPN providers offer servers in many locations worldwide. Because connections are fast, VPN is ideal for streaming geo-restricted media content
  3. Ideal for P2P filesharing – while many providers prohibit it, many are set up with filesharing in mind

VPN disadvantages

It is imperative that you choose a quality VPN service that does not store data or communications logs. In the event that a government agency demands the VPN provider to reveal logs, users would be exposed. Furthermore, it is important that the VPN service implements proper load balancing and server randomization so that users always connect to a different VPN server.
  1. The VPN provider can see your internet activity – and in many countries is required by law to keep records of it, which may be handed over to the authorities or to copyright lawyers.
  2. VPNs are also vulnerable to server raids by the police, in an effort to obtain the information they may contain. This is why it is vital to choose a provider who keeps no logs (and is in a position to keep this promise). Of course, even when a VPN provider promises to keep no logs, you must trust them to keep their word …
  3. Costs money (although typically under $10 a month, or less if you buy in bulk)

What is Tor?

Tor is free software for enabling anonymous communication. The name is an acronym derived from the original software project name The Onion Router. Tor directs Internet traffic through a free, worldwide, volunteer network consisting of more than six thousand relays to conceal a user’s location and usage from anyone conducting network surveillance or traffic analysis. Using Tor makes it more difficult for Internet activity to be traced back to the user: this includes “visits to Web sites, online posts, instant messages, and other communication forms”. Tor’s use is intended to protect the personal privacy of users, as well as their freedom and ability to conduct confidential communication by keeping their Internet activities from being monitored. Ultimate anonymity - blackMORE Ops - 4
Onion routing is implemented by encryption in the application layer of a communication protocol stack, nested like the layers of an onion. Tor encrypts the data, including the destination IP address, multiple times and sends it through a virtual circuit comprising successive, randomly selected Tor relays. Each relay decrypts a layer of encryption to reveal only the next relay in the circuit in order to pass the remaining encrypted data on to it. The final relay decrypts the innermost layer of encryption and sends the original data to its destination without revealing, or even knowing, the source IP address. Because the routing of the communication is partly concealed at every hop in the Tor circuit, this method eliminates any single point at which the communicating peers can be determined through network surveillance that relies upon knowing its source and destination. It is a decentralized system that allows users to connect through a network of relays rather than making a direct connection. The benefit of this method is that your IP address is hidden from the sites you visit by bouncing your connection from server to server at random, in essence, losing the trail.

Tor advantages

  1. No-one can trace you to external IPs visited
  2. Distributed network – almost impossible to shut down or attack in a meaningful way
  3. Free

Tor disadvantages

While your data is encrypted along each of the relay nodes, the final connection point at the last relay in the chain can be compromised if the requested site does not use SSL. Tor has a known drawback of substantially slowing down your browsing because of the numerous hops your data is relayed through. For those concerned with the prying eyes of the government, Tor was created in conjunction with the U.S. Navy and is still used by many government agencies. Because Tor has been widely used by political dissidents, journalists, and even criminals, many governments are watchful of Tor users. This could potentially lead to you being earmarked as a criminal type and having all of your online activity monitored.
  1. Very slow – because your data is randomly bounced through a number of nodes, each of which could be anywhere in the world, using Tor can be painfully slow.
  2. Not suitable for P2P filesharing – while there is no way to stop you from using BitTorrent over Tor (and people do it) it is a) very slow, and b) very bad form as it slows down the entire network for every other user, for some of whom access to the internet via Tor may be of critical and possibly life threatening importance.
  3. While it can, at a pinch, be used for location spoofing, Tor is a very fiddly and inefficient way to go about it. In addition to this, the slowness of Tor means that using the service to stream geo-restricted media services is unfeasible.

Tor and VPN together

Advantages and disadvantages of using Proxy, VPN, TOR and TOR and VPN together - blackMORE Ops - 1The two can be used in conjunction with one another for an added layer of security, however this will dramatically slow down service due to Tor’s method of randomly sending the connection through many servers throughout the world. However, it is imperative to use a VPN so your data is encrypted in that a VPN is used to keep data secure. A VPN provider that does not keep logs is equally important.
The great advantage of Tor is that you do not need to trust anyone – your internet use is completely anonymised. However it is very slow, and largely as a consequence of this, it is not suitable for many of the most popular activities people want to use VPN for, such as filesharing and streaming geo-restricted media content.
As long as a trustworthy no logs VPN provider is used, then VPN is a very secure, consumer oriented privacy solution that provides much greater performance and flexibility than Tor can offer.
It should be noted that with either method users will suffer the combined speed hit of using both services together. There’s two variations of it:

Tor through VPN

In this configuration you connect first to your VPN server, and then to the Tor network, before accessing the internet:
Your computer -> VPN -> Tor -> internet
This is what happens when you use the Tor Browser (less safe) or Whonix (more safe) while connected to a VPN server, and means that your apparent IP on the internet is that of the Tor exit node.

Tor through VPN advantages

  1. Your ISP will not know that you are using Tor (although they will know that you are using VPN)
  2. If your traffic is being monitored by a malicious Tor exit node then it less likely that your VPN provider will keep logs than your ISP. It therefore provides an additional level of privacy, but not anonymity.

Tor through VPN disadvantages

  1. A malicious Tor exit node will still be able to monitor your internet activity and trace it back to your VPN provider. Good VPN providers promise to keep no logs of users’ activities, and use shared IP addresses. If these promises are kept then this will provide a very effective second layer of protection, but it does rely entirely on trusting your VPN provider (as does regular VPN use).
  2. Tor exit nodes are often blocked.
So basically, this configuration hides Tor use from your ISP, and if using a trustworthy logless VPN service can provide a meaningful extra layer of security. Unfortunately there is no way of ever guaranteeing that a VPN provider is 100 per cent trustworthy, and even the best of them can be compelled by a subpoena or court order to start real-time logging of an individual’s actions (although this would require that law enforcement is actively seeking that individual).
Journalist and whistleblowers who are only concerned about law enforcement in their own country, and are not facing an adversary with an international reach, should find this setup very effective if using an overseas VPN provider, but should remember that some governments are not above taking extra-legal measures to obtain the information they want.

VPN through Tor

This involves connecting first to Tor, and then through a VPN server to the internet:
Your computer -> encrypt with VPN -> Tor -> VPN -> internet
This setup requires you to configure your VPN client to work with Tor, and the only VPN providers we know of to support this are the excellent AirVPN and BolehVPN. Your apparent IP on the internet is that of the VPN server.

VPN through Tor advantagesUltimate anonymity - blackMORE Ops - 2

  1. Because you connect to the VPN server through Tor, the VPN provider cannot ‘see’ your real IP address – only that of the Tor exit node. When combined with an anonymous payment method (such as properly mixed Bitcoins) made anonymously over Tor, this means the VPN provider has no way of identifying you, even if it did keep logs.
  2. Protection from malicious Tor exit nodes, as data is encrypted by the VPN client before entering (and exiting) the Tor network (although the data is encrypted, your ISP will be able to see that it is heading towards a Tor node).
  3. Bypasses any blocks on Tor exit nodes.
  4. Allows you to choose server location (great for geo-spoofing).

VPN through Tor disadvantages

  1. None really, apart from the speed hit, and that setup can be a little more complex.
  2. Slightly more vulnerable to global end-to-end timing attack.
  3. To access Tor hidden services you need to run the Tor browser. This introduces yet another layer of obfuscation, but will slow the connection down even further.
As you can see, this is by far the better configuration. With care, you can maintain true anonymity while benefiting from the extra layer of protection afforded by using VPN (with the side-benefit of not suffering from blocked Tor exit nodes.)

Conclusion

In conclusion, a proxy server is completely browser based, and is not as compatible with certain web pages that use non-browser technology. However, a VPN will work with ALL internet based services, but will offer less choice on which applications will get run through your ISP, as with the VPN, everything is routed through that server when connected.
Having all that in mind, if you are a mafia Whistle-blower, or a dissident living under regime in which it would be very dangerous to be caught performing certain activity (such as writing political blogs), then VPN through Tor is the ultimate safest solution as at that point you are almost certainly going via solid encryption and will help you browse the web absolutely anonymously. Yes, Even it means you need to spend a few coins more, the result you get will be stellar and it is worth it!
Just remember that there is no such thing as a 100% guarantee of anonymity, whichever route you take. There are always, at least potentially, loopholes in any security arrangement, and individuals often give their true identifies away through patterns of behaviors such browsing pattern, super cookies, unguarded comments made while on-line, or any other number of simple and not-so-simple mistakes. But as far as I’ve researched, there’s no match for VPN through Tor at this point. I personally use PrivateInternetAccess for VPN service as they are yet to be proven otherwise. Instruction for setting up PIA can be found here.
Do you think otherwise? Let us know via comment section.

Thursday, March 17, 2016

How Linux CPU usage time and Percentage can be calculated

https://github.com/Leo-G/DevopsWiki/wiki/How-Linux-CPU-usage-time-and-Percentage-can-be-calculated

Overview

CPU time is allocated in discrete time slices (ticks). For a certain number of time slices, the cpu is busy, other times it is not (which is represented by the idle process). In the picture below the CPU is busy for 6 of the 10 CPU slices. 6/10 = .60 = 60% of busy time (and there would therefore be 40% idle time).

Note: A tick(cycle) is the time it takes to send a single pulse. A pulse consists of a high voltage followed by a low voltage. There can be billions of ticks per second depending on the frequency(GHz) of the CPU clock.
You can get the number of CPU ticks since boot from /proc/stat
 cat /proc/stat
     user nice system idle iowait  irq  softirq steal guest guest_nice
cpu  4705 356  584    3699   23    23     0       0     0          0
  • user: normal processes executing in user mode
  • nice: niced processes executing in user mode
  • system: processes executing in kernel mode
  • idle: twiddling thumbs
  • iowait: waiting for I/O to complete
  • irq: servicing interrupts
  • softirq: servicing softirqs
  • steal: involuntary wait
  • guest: running a normal guest
  • guest_nice: running a niced guest

Formula

To calculate Linux CPU usage time subtract the idle CPU time from the total CPU time as follows:
Total CPU time since boot = user+nice+system+idle+iowait+irq+softirq+steal
Total CPU Idle time since boot = idle + iowait
Total CPU usage time since boot = Total CPU time since boot - Total CPU Idle time since boot
Total CPU percentage = Total CPU usage time since boot/Total CPU time since boot X 100
If you use the formula on the example data above you should get a Linux CPU usage Percentage of 60%.
Note: Guest and Guest_nice are already accounted in user and nice, hence they are not included in the total calculation
For real time CPU usage, you will need to calculate the time between two intervals.
Below is an example of a Bash Script by Paul Colby that does this
    #!/bin/bash
    # by Paul Colby (http://colby.id.au), no rights reserved ;)

    PREV_TOTAL=0
    PREV_IDLE=0

    while true; do
      # Get the total CPU statistics, discarding the 'cpu ' prefix.
      CPU=(`sed -n 's/^cpu\s//p' /proc/stat`)
      IDLE=${CPU[3]} # Just the idle CPU time.

      # Calculate the total CPU time.
      TOTAL=0
      for VALUE in "${CPU[@]}"; do
        let "TOTAL=$TOTAL+$VALUE"
      done

      # Calculate the CPU usage since we last checked.
      let "DIFF_IDLE=$IDLE-$PREV_IDLE"
      let "DIFF_TOTAL=$TOTAL-$PREV_TOTAL"
      let "DIFF_USAGE=(1000*($DIFF_TOTAL-$DIFF_IDLE)/$DIFF_TOTAL+5)/10"
      echo -en "\rCPU: $DIFF_USAGE%  \b\b"

      # Remember the total and idle CPU times for the next check.
      PREV_TOTAL="$TOTAL"
      PREV_IDLE="$IDLE"

      # Wait before checking again.
      sleep 1
    done
Ref https://www.kernel.org/doc/Documentation/filesystems/proc.txt section 1.8
https://github.com/pcolby/scripts/blob/master/cpu.sh
http://stackoverflow.com/questions/23367857/accurate-calculation-of-cpu-usage-given-in-percentage-in-linux
http://serverfault.com/questions/648704/how-are-cpu-time-and-cpu-usage-the-same
http://www.webopedia.com/TERM/C/clock_tick.html
http://www.pcworld.com/article/221559/cpu.html
http://stackoverflow.com/questions/16726779/how-do-i-get-the-total-cpu-usage-of-an-application-from-proc-pid-stat
http://www.ask.com/technology/many-times-system-clock-tick-per-second-1-ghz-processor-b9028ab0b0de7883
https://github.com/torvalds/linux/blob/master/fs/proc/stat.c

Friday, January 22, 2016

How to setup a intermediate compatible SSL website with LetsEncrypt certificate

https://www.howtoforge.com/tutorial/how-to-setup-intermediate-compatible-ssl-website-with-letsencrypt-certificate

Let’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit.
The key principles behind Let’s Encrypt are:
  • Free: Anyone who owns a domain name can use Let’s Encrypt to obtain a trusted certificate at zero cost.
  • Automatic: Software running on a web server can interact with Let’s Encrypt to painlessly obtain a certificate, securely configure it for use, and automatically take care of renewal.
  • Secure: Let’s Encrypt will serve as a platform for advancing TLS security best practices, both on the CA side and by helping site operators properly secure their servers.
  • Transparent: All certificates issued or revoked will be publicly recorded and available for anyone to inspect.
  • Open: The automatic issuance and renewal protocol will be published as an open standard that others can adopt.
  • Cooperative: Much like the underlying Internet protocols themselves, Let’s Encrypt is a joint effort to benefit the community, beyond the control of any one organization.
(source: https://letsencrypt.org/about/)

Intro:

First, we have to mention about some dark sides of Let's Encrypt service. However great the idea of free public and open certificate authority is, it brings also many troubles for us. Developers have tried to make the system of obtaining certificates as simple as possible, but it still requires a higher skill of server administering. Therefore, many of developers like the one from ISPConfig (http://www.ispconfig.org/) have implemented this solution directly into their. This brings people more effective deployment and supervision over the whole system much easier and flexible.

Real complication:

Many people have decided to implement Let's Encrypt into their production sites. I find this still a very bad idea to be done without being very (but really very) careful. Let's Encrypt brings you freedom but also limits you in using the certificate with SHA-256 RSA Encryption. Support for SHA-2 has improved over the last few years. Most browsers, platforms, mail clients and mobile devices already support SHA-2. However, some older operating systems such as Windows XP pre-SP3 do not support SHA-2 encryption. Many organizations will be able to convert to SHA-2 without running into user experience issues, and many may want to encourage users running older, less secure systems to upgrade.
In this tutorial, we are going to deal with this incompatibility in a simple, but still nasty way.

Prerequisites:

  • Apache version 2.4 and higher
  • OpenSSL version 1.0.1e and higher
  • Apache mod_rewrite enabled

The whole idea:

As mentioned before, there are still devices incompatible with SHA-256 signature in the Internet. When I was forced to deploy an SSL to some websites, I had to decide from two options:
  1. Using Let's Encrypt having it for free but not for all
  2. Buying a certificate with 128 bit signature
Well, still the option no. 1 was the only way as it was promised to customer long days ago (:

No more theory:

I hope I have explained the needed and now we can deal with the unsupported viewers of our website. There are many people using Windows XP machines with SP2 and lower (Yes, there are still plenty of them). So we have to filter these people.
In your “/etc/apache2/sites-available/your_domain.com.conf” add following on the end of the file:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !(NT\ 5) [NC]
RewriteRule ^(.*) https:// your_domain.com [R]
RewriteCond gets a string from http header of the guest accessing your page. You can simply check yours and find more information here: http://www.useragentstring.com/
The condition we used tells us something like “if string doesn't contain 'NT 5'” then RewriteRule executes/applies the rule of redirecting [R] to https variant of your domain, NT 5 is a OS version string for Windows XP devices.
If you don't use this redirection, incompatible users won't be able to access your https website.
I have to warn you this solution is not 100% perfect as some of guest doesn't have to provide you relevant or real information. I have worked with AWstats to figure out what rate of unknown systems are accessing my page and it is about 1.3%, so pretty few requests. If you want to deal with unknown operating systems to ensure their compatibility, you can add unknown in the condition as well (RewriteCond %{HTTP_USER_AGENT} !(NT\ 5|unknown) [NC]).
AWstats:
Awstats graphic.
After successfully “non-redirecting” your incompatible visitors (keeping them in http insecure world) you can focus on https side.

HTTPS configuration:

Now we assume you already assigned the certificate to your web server and also enabled it.
In your vhost config file again, add following:
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA 
SSLProtocol All -SSLv2 -SSLv3 
SSLCompression off 
SSLHonorCipherOrder On 
The CipherSuite used here is a bit longer than usual. It's for better compatibility. You can get your own from: https://cipherli.st/ or https://mozilla.github.io/server-side-tls/ssl-config-generator/
I must again mention, you wont ever get a perfect configuration to meet the high security policy and also compatibility. You should find a compromise.
After using these settings, you can test your server configuration and compatibility at: https://www.ssllabs.com/ssltest/index.html
You are going to find a long list of compatible devices and the incompatible ones, also some more information to point you for your own “perfect” solution.