Amazon S3 Faq

Copy a bucket
Using AWS CLI:
Test it with:
aws s3 sync --dryrun s3://from/this/bucket/ s3://to/this/bucket/
Do it with:
aws s3 sync s3://from/this/bucket/ s3://to/this/bucket/

Creating an Intellij IDEA python Project

Requires the python plugin to be installed.
I am using IDEA 14.1.5

1. File -> New -> Project
2. Python -> Next
3. Next
4. Choose root path -> Finish
5. Tools -> Manage Python Packages
6. Add packages you need.

Intellij IDEA node.js

Intellij IDEA is a decent IDE to use with node.js.

Just download the plugin from the makers of IDEA to use it.

To use 3rd party node.js libraries I have found these 2 ways:

1. In the terminal use npm to install the library.
2. Use the IDEA GUI and do:

• File -> Settings -> Languages & Frameworks -> Node.js and NPM
• Click the green + on the right side
• Choose the library 'package'
• Install Package

IIS cannot show files on Google Drive

The problem:

IIS cannot show files on Google Drive

Is due to file permissions.

You might be able to change the sites Authentication -> Anonymous Authentication to be a user on the machine.

But this works:

On the folder with the web site in the properties security section add this user

MACHINE_NAME\IIS_IUSRS

And make sure it has read permission.

Don't let environment values persist after a batch file runs

To prevent environment variables declared in a batch from being persisted after the batch file is run, use this command before setting them:

setlocal

Reference: http://ss64.com/nt/setlocal.html

Fiddler FAQ

Headers, How to add headers

1. Run fiddler
2. Use the editor
3. Find the method:    static function OnBeforeRequest(oSession: Session) {
4. At the end of it add your handler code.
5. Save the changes and try it (seems like fiddler picks up the changes when you save)
6. See the sample here:

        if (oSession.fullUrl.Contains("/somepartOfUrlToMatch"))

        {

            FiddlerObject.alert('show alert');

            oSession.oRequest["X-MY-USER"]="someUsername";

            oSession.oRequest["X-MY-ROLE"]="someRole";

        }

Installing it:

1. Get it from telrick http://www.telerik.com/fiddler
2. Install it.
3. Run it
4. Try to "Rules -> Customize Rules" and then have it download the editor for you (or if you are a masochist edit in the default editor)
5. Install the editor
6. Use the editor to adjust the rules (it seems to automatically know where the file is and to automatically load it)

Popup, How to show a debug popup:
FiddlerObject.alert('show this message');

Java -jar debug with -Xdebug

I had trouble attaching when running a jar via:
java -jar

Seems like all the param after -jar are parameters to it.
So I had to put the -Xdebug _before_ the -jar flag.

vi / vim FAQ

Replace 'substitution':
:%s/Orig_Text/New_Text/g

% makes it affect all lines (instead of only the current line)
g makes it happen on all occurrences on each line (instead of just the first)

ssh FAQ

Disconnect - Stop ssh shell from timing out, see here:
http://www.howtogeek.com/howto/linux/keep-your-linux-ssh-session-from-disconnecting/

.ssh/config - Use this to remember things for you
An a example:

Host debug1profile
    HostName debug1.fake.com
   User fakeuserNameToLoginAs
   IdentityFile /xxx/keyToUseWithThatHost.pem

With a config like that just do this to login (it grabs all the needed values from the config file)

ssh debug1profile

Windows Find how it can do something like grep

Check to see how port 80 is being used:
netstat -noa | find ":80"

Fancy watch like monitoring (at least fancy for windows):
for /L %i in (0,0,0) do @cls && netstat -ano|find ":80" && @ping -n 1 -w 2000 224.0.0.0 >NUL && @cls


See http://digitalcrunch.com/spyware-removal/monitor-ports-on-windows/

Java FAQ

Remote Debugging, what to add on the command line to allow it:

-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

PostgreSQL Allow Connections from Everywhere

PostgreSQL doesn't normally allow connections from everywhere.
But for development you might want to enable it.

On Windows, a typical path to the file to edit is like:
C:\Program Files\PostgreSQL\9.4\data\pg_hba.conf

Add a line like this to the end of that file:
host    all             all             0.0.0.0/0            md5

Also make sure your firewall allows remote machines to get in, or it won't matter if _only_ postgres allows them.

Java executable JAR: What class does it run?

I wanted to unpack a jar file and run it so I could tweak some properties.
I found I had to use jar to unpack it (zip had name collisions).
After unpacking look here to find out what class to run:

META-INF/MANIFEST.MF

Look at the value of:
Main-Class:

It is the class to run.

Just use java to run it:

java -cp PATH_TO_UNPACKED_FILES FULLY_QUALIFIED_PATH_OF_CLASS_TO_RUN

I found this guide to be a nice simple intro to Hibernate:

http://docs.jboss.org/hibernate/orm/4.3/quickstart/en-US/html_single/

Linux Package Manager List

Centos: yum

Ubuntu: apt-get

C++ Debug when a public double field is being set

C++ Debug when a public double field is being set.

The problem:

• There is a public field that is a double
• It gets set a lot
• You need to know when it gets a certain value
• Or You need to know when it's value is used or set

The solution:

Replace 'double' with 'DebugableDouble'.

Since it has an equals operator that takes a double, a recompile will use it.

So you don't have to change all of the places it is used to make them use setters and getters.

Here is the class:

class DebugableDouble
{
private:
double m_val;

public:

DebugableDouble(double val)
{
m_val = val;
}

operator double()
{
return m_val;
}

double operator = (double newVal)
{
m_val = newVal;
return m_val;
}
};

Intellij IDEA FAQ

Work in progress...

Classpath order can be changed:

1. File -> Project Structure... -> Modules
2. Choose the module the classpath needs to be adjusted for
3. Drag the item to where it should be (top is at the beginning)
4. If (really when) it the apply button doesn't enable, use the up and down arrows to move it and enable apply
5. apply

Open/Reopen on startup

That it opens the last project automatically on start up can be disabled.
File -> Settings
search for 'start'
Use the checkbox for:
Reopen last project on startup

Parameters - To see what parameters a method wants, type control-P

VM Options for running IDEA

1. I found the file for 64 bit here: "C:\Program Files (x86)\JetBrains\IntelliJ IDEA 14.1.5\bin\idea64.exe.vmoptions"
2. Edit it with an admin notepad or something

Intellij IDEA Font Size

I made these bindings:

1. Font size up -> control - numpad 8
2. Font size down -> control - numpad 2
3. Font size reset -> control - numpad 5

To do so I went to "File -> Settings ... Keymap" then in search box typed "font", right clicked on the action to set the keys.

It warns you if the key is already in use.

Run a Maven test only when a certain Profile exists

I had a test that would only run on a different OS than my main work, and beyond that it really should only run from a docker container.

This shows how to make it not run by default, but to only run with this maven option set on the command line: -Pnative-test
Here is a sample command line:
mvn -o surefire:test -Pnative-test

To set this up, add this to your pom (the includes can be wildcarded):

<profiles>
    <profile>
        <id>native-test</id>
        <activation>
            <property>
                <name>native-test</name>
            </property>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <includes>
                            <include>**/RunOnlyWithProfile-NativeT.java</include>
                        </includes>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

edit: updated tag

apt FAQ

apt FAQ

Work in Progress...

Search for something apt can install

For example boost:

aptitude search libboost

Search, refresh search path

sudo aptitude update

Reference

http://stackoverflow.com/questions/5759434/boost-installation

Port Forwarding FAQ

Port Forwarding FAQ

Work in Progress...

Machine forwards a port from source machine:

ssh -L 8788:localhost:8787 USER_NAME@IP_OF_SOURCE_MACHINE

Maven FAQ

Maven FAQ

Work in Progress...


Documentation, generate it
mvn site

Debug tests
mvn -Dmaven.surefire.debug test

Debug tests with jvm settings
mvn -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE" test

Dependencies:
mvn dependency:resolve
mvn dependency:tree

Do Not Run Other Phases, just test only:
mvn -Dtest=NameOfTest surefire:test

Effective pom, how to show it:
mvn help:effective-pom

Multiple tests, Run only a limited set of tests:
mvn -Dtest=TestSquare,TestCi*le test

Repo: Search for things in the 'official' repo
https://repository.sonatype.org/#welcome

Single test, Run only a Single test
mvn -Dtest=TestCircle test

References:
http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html
http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html
http://stackoverflow.com/questions/14881730/run-junit-tests-in-maven-without-building-and-copying-files

Docker FAQ

Docker FAQ

Work in progress...

Load a docker image from a tar file:
docker load < TARGET_FILENAME.tar
See Save to find how to make something to load

Remove (delete) ALL existing containers:

docker rm $(docker ps -aq)

Runing from docker shows something like this:
strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Permission denied

strace: test_ptrace_setoptions_for_all: unexpected exit status 1

Perhaps docker needs to be run with the option --privileged

Save a docker image to a tar file (instead of putting in an archive):
docker save IMAGE_NAME > TARGET_FILENAME.tar
See Load for what to do with a save

Start a bash shell inside a running docker container:
Find the name of the container:
docker ps
or depending how your system is setup:
sudo docker ps
Then with the name (replacing 'crazy_name' in the examples) start interactive bash like this:
docker exec -it crazy_name bash
or:
sudo docker exec -it crazy_name bash

Terms

container (think of this as a running exe)

• Running instance of an image.
• It can be actively running and shows up via:
• docker ps
• It can be sort of sleeping and shows up when _all_ docker containers are shown via:
• docker ps -a
• Can be made of a an image via:
• docker commit

image (think of this like an exe file on disk)

• Stored set of stuff that must be run to become a container.
• Can be seen via
• docker images
• A container can be made into an image via:
• docker commit

Maven Tests

Run a Single Test:
mvn -Dtest=TestCircle test

Skip Tests:
mvn install -DskipTests

Reference:

http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html

Ubuntu Package Stuff

I may update this with more later

Search for a package:
http://packages.ubuntu.com/

See what packages are installed:
dpkg --get-selections
And grep to filter

Installing the JDK and maven on Ubuntu Server

This worked for me:

sudo apt-get install maven

sudo apt-get install openjdk-7-jdk


Or was it in the opposite order?


Reference:
http://askubuntu.com/questions/117189/apt-get-install-openjdk-7-jdk-doesnt-install-javac-why

Favorite vim Settings

Put these in the file:
~/.vimrc

I don't like the syntax highlighting to use dark colors:

set background=dark

Hmm, only one for now, I bet it will become a list.

Ubuntu on VirtualBox with Dual Network Interfaces

I needed to be able to reach the internet from a Ubuntu VM, and be able to get into it from my host machine, and I needed its IP address to be fixed.

I could not use the Bridged Adapter setting because the VM was going to be part of my work domain, and I found that if I shut it down for a while, or switched snapshots it would occasionally change its IP address.

A configuration that works for me is to make it have 2 adapters.

In VirtualBox Manager I went to:
File -> Preferences -> Network -> Host-only Networks
Verified there was an entry and set it up to have a DHCP server.

In my VM settings I set up so that network adapter 1 was host only, and 2 was NAT.

I then installed Ubuntu Server.

I told it to use the 2nd adapter for internet.

After it started I did:
vi /etc/network/interfaces
Which only showed:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
iface eth1 inet dhcp

So I tacked this onto the end of it:

auto eth0

iface eth0 inet dhcp

And then rebooted.

Now it can ping the host (Windows in my case), and the host can ping it, and I can ssh into it.

Updating Ubunu Server

Looks like these steps are good for updating a Ubuntu Server:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

And do a yes 'y' to everything.

They can be run multiple times to be sure.

Reference: http://askubuntu.com/questions/196768/how-to-install-updates-via-command-line

Setting Up

Setting up...