SyntaxFix.com - Programming Questions & Answers Hub For Beginners


Some Of The Best Answers From Latest Asked Questions

Datatable vs Dataset

There are some optimizations you can use when filling a DataTable, such as calling BeginLoadData(), inserting the data, then calling EndLoadData(). This turns off some internal behavior within the DataTable, such as index maintenance, etc. See this article for further details.

How to call shell commands from Ruby

Using the answers here and linked in Mihai's answer, I put together a function that meets these requirements:

  1. Neatly captures STDOUT and STDERR so they don't "leak" when my script is run from the console.
  2. Allows arguments to be passed to the shell as an array, so there's no need to worry about escaping.
  3. Captures the exit status of the command so it is clear when an error has occurred.

As a bonus, this one will also return STDOUT in cases where the shell command exits successfully (0) and puts anything on STDOUT. In this manner, it differs from system, which simply returns true in such cases.

Code follows. The specific function is system_quietly:

require 'open3'

class ShellError < StandardError; end

#actual function:
def system_quietly(*cmd)
  exit_status=nil
  err=nil
  out=nil
  Open3.popen3(*cmd) do |stdin, stdout, stderr, wait_thread|
    err = stderr.gets(nil)
    out = stdout.gets(nil)
    [stdin, stdout, stderr].each{|stream| stream.send('close')}
    exit_status = wait_thread.value
  end
  if exit_status.to_i > 0
    err = err.chomp if err
    raise ShellError, err
  elsif out
    return out.chomp
  else
    return true
  end
end

#calling it:
begin
  puts system_quietly('which', 'ruby')
rescue ShellError
  abort "Looks like you don't have the `ruby` command. Odd."
end

#output: => "/Users/me/.rvm/rubies/ruby-1.9.2-p136/bin/ruby"

Creating a custom JButton in Java

Yes, this is possible. One of the main pros for using Swing is the ease with which the abstract controls can be created and manipulates.

Here is a quick and dirty way to extend the existing JButton class to draw a circle to the right of the text.

package test;

import java.awt.Color;
import java.awt.Container;
import java.awt.Dimension;
import java.awt.FlowLayout;
import java.awt.Graphics;

import javax.swing.JButton;
import javax.swing.JFrame;

public class MyButton extends JButton {

    private static final long serialVersionUID = 1L;

    private Color circleColor = Color.BLACK;

    public MyButton(String label) {
        super(label);
    }

    @Override
    protected void paintComponent(Graphics g) {
        super.paintComponent(g);

        Dimension originalSize = super.getPreferredSize();
        int gap = (int) (originalSize.height * 0.2);
        int x = originalSize.width + gap;
        int y = gap;
        int diameter = originalSize.height - (gap * 2);

        g.setColor(circleColor);
        g.fillOval(x, y, diameter, diameter);
    }

    @Override
    public Dimension getPreferredSize() {
        Dimension size = super.getPreferredSize();
        size.width += size.height;
        return size;
    }

    /*Test the button*/
    public static void main(String[] args) {
        MyButton button = new MyButton("Hello, World!");

        JFrame frame = new JFrame();
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(400, 400);

        Container contentPane = frame.getContentPane();
        contentPane.setLayout(new FlowLayout());
        contentPane.add(button);

        frame.setVisible(true);
    }

}

Note that by overriding paintComponent that the contents of the button can be changed, but that the border is painted by the paintBorder method. The getPreferredSize method also needs to be managed in order to dynamically support changes to the content. Care needs to be taken when measuring font metrics and image dimensions.

For creating a control that you can rely on, the above code is not the correct approach. Dimensions and colours are dynamic in Swing and are dependent on the look and feel being used. Even the default Metal look has changed across JRE versions. It would be better to implement AbstractButton and conform to the guidelines set out by the Swing API. A good starting point is to look at the javax.swing.LookAndFeel and javax.swing.UIManager classes.

http://docs.oracle.com/javase/8/docs/api/javax/swing/LookAndFeel.html

http://docs.oracle.com/javase/8/docs/api/javax/swing/UIManager.html

Understanding the anatomy of LookAndFeel is useful for writing controls: Creating a Custom Look and Feel

Convert HashBytes to VarChar

I have found the solution else where:

SELECT SUBSTRING(master.dbo.fn_varbintohexstr(HashBytes('MD5', 'HelloWorld')), 3, 32)

What are MVP and MVC and what is the difference?

I have used both MVP and MVC and although we as developers tend to focus on the technical differences of both patterns the point for MVP in IMHO is much more related to ease of adoption than anything else.

If I’m working in a team that already as a good background on web forms development style it’s far easier to introduce MVP than MVC. I would say that MVP in this scenario is a quick win.

My experience tells me that moving a team from web forms to MVP and then from MVP to MVC is relatively easy; moving from web forms to MVC is more difficult.

I leave here a link to a series of articles a friend of mine has published about MVP and MVC.

http://www.qsoft.be/post/Building-the-MVP-StoreFront-Gutthrie-style.aspx

How do I create a branch?

svn cp /trunk/ /branch/NEW_Branch

If you have some local changes in trunk then use Rsync to sync changes

rsync -r -v -p --exclude ".svn" /trunk/ /branch/NEW_Branch

What do the result codes in SVN mean?

For additional details see the SVNBook: "Status of working copy files and directories".

The common statuses:

U: Working file was updated

G: Changes on the repo were automatically merged into the working copy

M: Working copy is modified

C: This file conflicts with the version in the repo

?: This file is not under version control

!: This file is under version control but is missing or incomplete

A: This file will be added to version control (after commit)

A+: This file will be moved (after commit)

D: This file will be deleted (after commit)

S: This signifies that the file or directory has been switched from the path of the rest of the working copy (using svn switch) to a branch

I: Ignored

X: External definition

~: Type changed

R: Item has been replaced in your working copy. This means the file was scheduled for deletion, and then a new file with the same name was scheduled for addition in its place.

L : Item is locked

E: Item existed, as it would have been created, by an svn update.

Python: What OS am I running on?

You can also use sys.platform if you already have imported sys and you don't want to import another module

>>> import sys
>>> sys.platform
'linux2'

Learning to write a compiler

If you're willing to use LLVM, check this out: http://llvm.org/docs/tutorial/. It teaches you how to write a compiler from scratch using LLVM's framework, and doesn't assume you have any knowledge about the subject.

The tutorial suggest you write your own parser and lexer etc, but I advise you to look into bison and flex once you get the idea. They make life so much easier.

What good technology podcasts are out there?

My favorites:

  1. thirsty developer
  2. pixel8d
  3. stackoverflow
  4. dotnetrocks
  5. alt.net podcast
  6. codecast
  7. hanselminutes

cheers

How do you express binary literals in Python?

For reference—future Python possibilities:
Starting with Python 2.6 you can express binary literals using the prefix 0b or 0B:

>>> 0b101111
47

You can also use the new bin function to get the binary representation of a number:

>>> bin(173)
'0b10101101'

Development version of the documentation: What's New in Python 2.6

Make XAMPP / Apache serve file outside of htdocs folder

You can set Apache to serve pages from anywhere with any restrictions but it's normally distributed in a more secure form.

Editing your apache files (http.conf is one of the more common names) will allow you to set any folder so it appears in your webroot.

EDIT:

alias myapp c:\myapp\

I've edited my answer to include the format for creating an alias in the http.conf file which is sort of like a shortcut in windows or a symlink under un*x where Apache 'pretends' a folder is in the webroot. This is probably going to be more useful to you in the long term.

How to check for file lock?

Instead of using interop you can use the .NET FileStream class methods Lock and Unlock:

FileStream.Lock http://msdn.microsoft.com/en-us/library/system.io.filestream.lock.aspx

FileStream.Unlock http://msdn.microsoft.com/en-us/library/system.io.filestream.unlock.aspx

How big can a MySQL database get before performance starts to degrade

I would focus first on your indexes, than have a server admin look at your OS, and if all that doesn't help it might be time for a master/slave configuration.

That's true. Another thing that usually works is to just reduce the quantity of data that's repeatedly worked with. If you have "old data" and "new data" and 99% of your queries work with new data, just move all the old data to another table - and don't look at it ;)

-> Have a look at partitioning.

How does database indexing work?

An index is just a data structure that makes the searching faster for a specific column in a database. This structure is usually a b-tree or a hash table but it can be any other logic structure.

Adding a Method to an Existing Object Instance

What Jason Pratt posted is correct.

>>> class Test(object):
...   def a(self):
...     pass
... 
>>> def b(self):
...   pass
... 
>>> Test.b = b
>>> type(b)
<type 'function'>
>>> type(Test.a)
<type 'instancemethod'>
>>> type(Test.b)
<type 'instancemethod'>

As you can see, Python doesn't consider b() any different than a(). In Python all methods are just variables that happen to be functions.

String literals and escape characters in postgresql

I find it highly unlikely for Postgres to truncate your data on input - it either rejects it or stores it as is.

milen@dev:~$ psql
Welcome to psql 8.2.7, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit

milen=> create table EscapeTest (text varchar(50));
CREATE TABLE
milen=> insert into EscapeTest (text) values ('This will be inserted \n This will not be');
WARNING:  nonstandard use of escape in a string literal
LINE 1: insert into EscapeTest (text) values ('This will be inserted...
                                              ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
INSERT 0 1
milen=> select * from EscapeTest;
          text
------------------------
 This will be inserted
  This will not be
(1 row)

milen=>

How do you debug PHP scripts?

XDebug is essential for development. I install it before any other extension. It gives you stack traces on any error and you can enable profiling easily.

For a quick look at a data structure use var_dump(). Don't use print_r() because you'll have to surround it with <pre> and it only prints one var at a time.

<?php var_dump(__FILE__, __LINE__, $_REQUEST); ?>

For a real debugging environment the best I've found is Komodo IDE but it costs $$.

Are PHP Variables passed by value or by reference?

For anyone who comes across this in the future, I want to share this gem from the PHP docs, posted by an anonymous user:

There seems to be some confusion here. The distinction between pointers and references is not particularly helpful. The behavior in some of the "comprehensive" examples already posted can be explained in simpler unifying terms. Hayley's code, for example, is doing EXACTLY what you should expect it should. (Using >= 5.3)

First principle: A pointer stores a memory address to access an object. Any time an object is assigned, a pointer is generated. (I haven't delved TOO deeply into the Zend engine yet, but as far as I can see, this applies)

2nd principle, and source of the most confusion: Passing a variable to a function is done by default as a value pass, ie, you are working with a copy. "But objects are passed by reference!" A common misconception both here and in the Java world. I never said a copy OF WHAT. The default passing is done by value. Always. WHAT is being copied and passed, however, is the pointer. When using the "->", you will of course be accessing the same internals as the original variable in the caller function. Just using "=" will only play with copies.

3rd principle: "&" automatically and permanently sets another variable name/pointer to the same memory address as something else until you decouple them. It is correct to use the term "alias" here. Think of it as joining two pointers at the hip until forcibly separated with "unset()". This functionality exists both in the same scope and when an argument is passed to a function. Often the passed argument is called a "reference," due to certain distinctions between "passing by value" and "passing by reference" that were clearer in C and C++.

Just remember: pointers to objects, not objects themselves, are passed to functions. These pointers are COPIES of the original unless you use "&" in your parameter list to actually pass the originals. Only when you dig into the internals of an object will the originals change.

And here's the example they provide:

<?php

//The two are meant to be the same
$a = "Clark Kent"; //a==Clark Kent
$b = &$a; //The two will now share the same fate.

$b="Superman"; // $a=="Superman" too.
echo $a;
echo $a="Clark Kent"; // $b=="Clark Kent" too.
unset($b); // $b divorced from $a
$b="Bizarro";
echo $a; // $a=="Clark Kent" still, since $b is a free agent pointer now.

//The two are NOT meant to be the same.
$c="King";
$d="Pretender to the Throne";
echo $c."\n"; // $c=="King"
echo $d."\n"; // $d=="Pretender to the Throne"
swapByValue($c, $d);
echo $c."\n"; // $c=="King"
echo $d."\n"; // $d=="Pretender to the Throne"
swapByRef($c, $d);
echo $c."\n"; // $c=="Pretender to the Throne"
echo $d."\n"; // $d=="King"

function swapByValue($x, $y){
$temp=$x;
$x=$y;
$y=$temp;
//All this beautiful work will disappear
//because it was done on COPIES of pointers.
//The originals pointers still point as they did.
}

function swapByRef(&$x, &$y){
$temp=$x;
$x=$y;
$y=$temp;
//Note the parameter list: now we switched 'em REAL good.
}

?>

I wrote an extensive, detailed blog post on this subject for JavaScript, but I believe it applies equally well to PHP, C++, and any other language where people seem to be confused about pass by value vs. pass by reference.

Clearly, PHP, like C++, is a language that does support pass by reference. By default, objects are passed by value. When working with variables that store objects, it helps to see those variables as pointers (because that is fundamentally what they are, at the assembly level). If you pass a pointer by value, you can still "trace" the pointer and modify the properties of the object being pointed to. What you cannot do is have it point to a different object. Only if you explicitly declare a parameter as being passed by reference will you be able to do that.

Why is Git better than Subversion?

The main points I like about DVCS are those :

  1. You can commit broken things. It doesn't matter because other peoples won't see them until you publish. Publish time is different of commit time.
  2. Because of this you can commit more often.
  3. You can merge complete functionnality. This functionnality will have its own branch. All commits of this branch will be related to this functionnality. You can do it with a CVCS however with DVCS its the default.
  4. You can search your history (find when a function changed )
  5. You can undo a pull if someone screw up the main repository, you don't need to fix the errors. Just clear the merge.
  6. When you need a source control in any directory do : git init . and you can commit, undoing changes, etc...
  7. It's fast (even on Windows )

The main reason for a relatively big project is the improved communication created by the point 3. Others are nice bonuses.

How do I use itertools.groupby()?

Sorting and groupby

from itertools import groupby

val = [{'name': 'satyajit', 'address': 'btm', 'pin': 560076}, 
       {'name': 'Mukul', 'address': 'Silk board', 'pin': 560078},
       {'name': 'Preetam', 'address': 'btm', 'pin': 560076}]


for pin, list_data in groupby(sorted(val, key=lambda k: k['pin']),lambda x: x['pin']):
...     print pin
...     for rec in list_data:
...             print rec
... 
o/p:

560076
{'name': 'satyajit', 'pin': 560076, 'address': 'btm'}
{'name': 'Preetam', 'pin': 560076, 'address': 'btm'}
560078
{'name': 'Mukul', 'pin': 560078, 'address': 'Silk board'}

How to create a new object instance from a Type

If this is for something that will be called a lot in an application instance, it's a lot faster to compile and cache dynamic code instead of using the activator or ConstructorInfo.Invoke(). Two easy options for dynamic compilation are compiled Linq Expressions or some simple IL opcodes and DynamicMethod. Either way, the difference is huge when you start getting into tight loops or multiple calls.

What is the difference between an int and an Integer in Java and C#?

(Java Version) In Simple words int is primitive and Integer is wrapper object for int.

One example where to use Integer vs int, When you want to compare and int variable again null it will throw error.

int a;
//assuming a value you are getting from data base which is null
if(a ==null) // this is wrong - cannot compare primitive to null
{
do something...}

Instead you will use,
Integer a;
//assuming a value you are getting from data base which is null
if(a ==null) // this is correct/legal
{ do something...}

The definitive guide to form-based website authentication

PART I: How To Log In

We'll assume you already know how to build a login+password HTML form which POSTs the values to a script on the server side for authentication. The sections below will deal with patterns for sound practical auth, and how to avoid the most common security pitfalls.

To HTTPS or not to HTTPS?

Unless the connection is already secure (that is, tunneled through HTTPS using SSL/TLS), your login form values will be sent in cleartext, which allows anyone eavesdropping on the line between browser and web server will be able to read logins as they pass through. This type of wiretapping is done routinely by governments, but in general, we won't address 'owned' wires other than to say this: Just use HTTPS.

In essence, the only practical way to protect against wiretapping/packet sniffing during login is by using HTTPS or another certificate-based encryption scheme (for example, TLS) or a proven & tested challenge-response scheme (for example, the Diffie-Hellman-based SRP). Any other method can be easily circumvented by an eavesdropping attacker.

Of course, if you are willing to get a little bit impractical, you could also employ some form of two-factor authentication scheme (e.g. the Google Authenticator app, a physical 'cold war style' codebook, or an RSA key generator dongle). If applied correctly, this could work even with an unsecured connection, but it's hard to imagine that a dev would be willing to implement two-factor auth but not SSL.

(Do not) Roll-your-own JavaScript encryption/hashing

Given the perceived (though now avoidable) cost and technical difficulty of setting up an SSL certificate on your website, some developers are tempted to roll their own in-browser hashing or encryption schemes in order to avoid passing cleartext logins over an unsecured wire.

While this is a noble thought, it is essentially useless (and can be a security flaw) unless it is combined with one of the above - that is, either securing the line with strong encryption or using a tried-and-tested challenge-response mechanism (if you don't know what that is, just know that it is one of the most difficult to prove, most difficult to design, and most difficult to implement concepts in digital security).

While it is true that hashing the password can be effective against password disclosure, it is vulnerable to replay attacks, Man-In-The-Middle attacks / hijackings (if an attacker can inject a few bytes into your unsecured HTML page before it reaches your browser, they can simply comment out the hashing in the JavaScript), or brute-force attacks (since you are handing the attacker both username, salt and hashed password).

CAPTCHAS against humanity

CAPTCHA is meant to thwart one specific category of attack: automated dictionary/brute force trial-and-error with no human operator. There is no doubt that this is a real threat, however, there are ways of dealing with it seamlessly that don't require a CAPTCHA, specifically properly designed server-side login throttling schemes - we'll discuss those later.

Know that CAPTCHA implementations are not created alike; they often aren't human-solvable, most of them are actually ineffective against bots, all of them are ineffective against cheap third-world labor (according to OWASP, the current sweatshop rate is $12 per 500 tests), and some implementations may be technically illegal in some countries (see OWASP Authentication Cheat Sheet). If you must use a CAPTCHA, use Google's reCAPTCHA, since it is OCR-hard by definition (since it uses already OCR-misclassified book scans) and tries very hard to be user-friendly.

Personally, I tend to find CAPTCHAS annoying, and use them only as a last resort when a user has failed to log in a number of times and throttling delays are maxed out. This will happen rarely enough to be acceptable, and it strengthens the system as a whole.

Storing Passwords / Verifying logins

This may finally be common knowledge after all the highly-publicized hacks and user data leaks we've seen in recent years, but it has to be said: Do not store passwords in cleartext in your database. User databases are routinely hacked, leaked or gleaned through SQL injection, and if you are storing raw, plaintext passwords, that is instant game over for your login security.

So if you can't store the password, how do you check that the login+password combination POSTed from the login form is correct? The answer is hashing using a key derivation function. Whenever a new user is created or a password is changed, you take the password and run it through a KDF, such as Argon2, bcrypt, scrypt or PBKDF2, turning the cleartext password ("correcthorsebatterystaple") into a long, random-looking string, which is a lot safer to store in your database. To verify a login, you run the same hash function on the entered password, this time passing in the salt and compare the resulting hash string to the value stored in your database. Argon2, bcrypt and scrypt store the salt with the hash already. Check out this article on sec.stackexchange for more detailed information.

The reason a salt is used is that hashing in itself is not sufficient -- you'll want to add a so-called 'salt' to protect the hash against rainbow tables. A salt effectively prevents two passwords that exactly match from being stored as the same hash value, preventing the whole database being scanned in one run if an attacker is executing a password guessing attack.

A cryptographic hash should not be used for password storage because user-selected passwords are not strong enough (i.e. do not usually contain enough entropy) and a password guessing attack could be completed in a relatively short time by an attacker with access to the hashes. This is why KDFs are used - these effectively "stretch the key", which means that every password guess an attacker makes causes multiple repetitions of the hash algorithm, for example 10,000 times, which causes the attacker to guess the password 10,000 times slower.

Session data - "You are logged in as Spiderman69"

Once the server has verified the login and password against your user database and found a match, the system needs a way to remember that the browser has been authenticated. This fact should only ever be stored server side in the session data.

If you are unfamiliar with session data, here's how it works: A single randomly-generated string is stored in an expiring cookie and used to reference a collection of data - the session data - which is stored on the server. If you are using an MVC framework, this is undoubtedly handled already.

If at all possible, make sure the session cookie has the secure and HTTP Only flags set when sent to the browser. The HttpOnly flag provides some protection against the cookie being read through XSS attack. The secure flag ensures that the cookie is only sent back via HTTPS, and therefore protects against network sniffing attacks. The value of the cookie should not be predictable. Where a cookie referencing a non-existent session is presented, its value should be replaced immediately to prevent session fixation.

Session state can also be maintained on the client side. This is achieved by using techniques like JWT (JSON Web Token).

PART II: How To Remain Logged In - The Infamous "Remember Me" Checkbox

Persistent Login Cookies ("remember me" functionality) are a danger zone; on the one hand, they are entirely as safe as conventional logins when users understand how to handle them; and on the other hand, they are an enormous security risk in the hands of careless users, who may use them on public computers and forget to log out, and who may not know what browser cookies are or how to delete them.

Personally, I like persistent logins for the websites I visit on a regular basis, but I know how to handle them safely. If you are positive that your users know the same, you can use persistent logins with a clean conscience. If not - well, then you may subscribe to the philosophy that users who are careless with their login credentials brought it upon themselves if they get hacked. It's not like we go to our user's houses and tear off all those facepalm-inducing Post-It notes with passwords they have lined up on the edge of their monitors, either.

Of course, some systems can't afford to have any accounts hacked; for such systems, there is no way you can justify having persistent logins.

If you DO decide to implement persistent login cookies, this is how you do it:

  1. First, take some time to read Paragon Initiative's article on the subject. You'll need to get a bunch of elements right, and the article does a great job of explaining each.

  2. And just to reiterate one of the most common pitfalls, DO NOT STORE THE PERSISTENT LOGIN COOKIE (TOKEN) IN YOUR DATABASE, ONLY A HASH OF IT! The login token is Password Equivalent, so if an attacker got their hands on your database, they could use the tokens to log in to any account, just as if they were cleartext login-password combinations. Therefore, use hashing (according to https://security.stackexchange.com/a/63438/5002 a weak hash will do just fine for this purpose) when storing persistent login tokens.

PART III: Using Secret Questions

Don't implement 'secret questions'. The 'secret questions' feature is a security anti-pattern. Read the paper from link number 4 from the MUST-READ list. You can ask Sarah Palin about that one, after her Yahoo! email account got hacked during a previous presidential campaign because the answer to her security question was... "Wasilla High School"!

Even with user-specified questions, it is highly likely that most users will choose either:

  • A 'standard' secret question like mother's maiden name or favorite pet

  • A simple piece of trivia that anyone could lift from their blog, LinkedIn profile, or similar

  • Any question that is easier to answer than guessing their password. Which, for any decent password, is every question you can imagine

In conclusion, security questions are inherently insecure in virtually all their forms and variations, and should not be employed in an authentication scheme for any reason.

The true reason why security questions even exist in the wild is that they conveniently save the cost of a few support calls from users who can't access their email to get to a reactivation code. This at the expense of security and Sarah Palin's reputation. Worth it? Probably not.

PART IV: Forgotten Password Functionality

I already mentioned why you should never use security questions for handling forgotten/lost user passwords; it also goes without saying that you should never e-mail users their actual passwords. There are at least two more all-too-common pitfalls to avoid in this field:

  1. Don't reset a forgotten password to an autogenerated strong password - such passwords are notoriously hard to remember, which means the user must either change it or write it down - say, on a bright yellow Post-It on the edge of their monitor. Instead of setting a new password, just let users pick a new one right away - which is what they want to do anyway. (An exception to this might be if the users are universally using a password manager to store/manage passwords that would normally be impossible to remember without writing it down).

  2. Always hash the lost password code/token in the database. AGAIN, this code is another example of a Password Equivalent, so it MUST be hashed in case an attacker got their hands on your database. When a lost password code is requested, send the plaintext code to the user's email address, then hash it, save the hash in your database -- and throw away the original. Just like a password or a persistent login token.

A final note: always make sure your interface for entering the 'lost password code' is at least as secure as your login form itself, or an attacker will simply use this to gain access instead. Making sure you generate very long 'lost password codes' (for example, 16 case-sensitive alphanumeric characters) is a good start, but consider adding the same throttling scheme that you do for the login form itself.

PART V: Checking Password Strength

First, you'll want to read this small article for a reality check: The 500 most common passwords

Okay, so maybe the list isn't the canonical list of most common passwords on any system anywhere ever, but it's a good indication of how poorly people will choose their passwords when there is no enforced policy in place. Plus, the list looks frighteningly close to home when you compare it to publicly available analyses of recently stolen passwords.

So: With no minimum password strength requirements, 2% of users use one of the top 20 most common passwords. Meaning: if an attacker gets just 20 attempts, 1 in 50 accounts on your website will be crackable.

Thwarting this requires calculating the entropy of a password and then applying a threshold. The National Institute of Standards and Technology (NIST) Special Publication 800-63 has a set of very good suggestions. That, when combined with a dictionary and keyboard layout analysis (for example, 'qwertyuiop' is a bad password), can reject 99% of all poorly selected passwords at a level of 18 bits of entropy. Simply calculating password strength and showing a visual strength meter to a user is good, but insufficient. Unless it is enforced, a lot of users will most likely ignore it.

And for a refreshing take on user-friendliness of high-entropy passwords, Randall Munroe's Password Strength xkcd is highly recommended.

Utilize Troy Hunt's Have I Been Pwned API to check users passwords against passwords compromised in public data breaches.

PART VI: Much More - Or: Preventing Rapid-Fire Login Attempts

First, have a look at the numbers: Password Recovery Speeds - How long will your password stand up

If you don't have the time to look through the tables in that link, here's the list of them:

  1. It takes virtually no time to crack a weak password, even if you're cracking it with an abacus

  2. It takes virtually no time to crack an alphanumeric 9-character password if it is case insensitive

  3. It takes virtually no time to crack an intricate, symbols-and-letters-and-numbers, upper-and-lowercase password if it is less than 8 characters long (a desktop PC can search the entire keyspace up to 7 characters in a matter of days or even hours)

  4. It would, however, take an inordinate amount of time to crack even a 6-character password, if you were limited to one attempt per second!

So what can we learn from these numbers? Well, lots, but we can focus on the most important part: the fact that preventing large numbers of rapid-fire successive login attempts (ie. the brute force attack) really isn't that difficult. But preventing it right isn't as easy as it seems.

Generally speaking, you have three choices that are all effective against brute-force attacks (and dictionary attacks, but since you are already employing a strong passwords policy, they shouldn't be an issue):

  • Present a CAPTCHA after N failed attempts (annoying as hell and often ineffective -- but I'm repeating myself here)

  • Locking accounts and requiring email verification after N failed attempts (this is a DoS attack waiting to happen)

  • And finally, login throttling: that is, setting a time delay between attempts after N failed attempts (yes, DoS attacks are still possible, but at least they are far less likely and a lot more complicated to pull off).

Best practice #1: A short time delay that increases with the number of failed attempts, like:

  • 1 failed attempt = no delay
  • 2 failed attempts = 2 sec delay
  • 3 failed attempts = 4 sec delay
  • 4 failed attempts = 8 sec delay
  • 5 failed attempts = 16 sec delay
  • etc.

DoS attacking this scheme would be very impractical, since the resulting lockout time is slightly larger than the sum of the previous lockout times.

To clarify: The delay is not a delay before returning the response to the browser. It is more like a timeout or refractory period during which login attempts to a specific account or from a specific IP address will not be accepted or evaluated at all. That is, correct credentials will not return in a successful login, and incorrect credentials will not trigger a delay increase.

Best practice #2: A medium length time delay that goes into effect after N failed attempts, like:

  • 1-4 failed attempts = no delay
  • 5 failed attempts = 15-30 min delay

DoS attacking this scheme would be quite impractical, but certainly doable. Also, it might be relevant to note that such a long delay can be very annoying for a legitimate user. Forgetful users will dislike you.

Best practice #3: Combining the two approaches - either a fixed, short time delay that goes into effect after N failed attempts, like:

  • 1-4 failed attempts = no delay
  • 5+ failed attempts = 20 sec delay

Or, an increasing delay with a fixed upper bound, like:

  • 1 failed attempt = 5 sec delay
  • 2 failed attempts = 15 sec delay
  • 3+ failed attempts = 45 sec delay

This final scheme was taken from the OWASP best-practices suggestions (link 1 from the MUST-READ list) and should be considered best practice, even if it is admittedly on the restrictive side.

As a rule of thumb, however, I would say: the stronger your password policy is, the less you have to bug users with delays. If you require strong (case-sensitive alphanumerics + required numbers and symbols) 9+ character passwords, you could give the users 2-4 non-delayed password attempts before activating the throttling.

DoS attacking this final login throttling scheme would be very impractical. And as a final touch, always allow persistent (cookie) logins (and/or a CAPTCHA-verified login form) to pass through, so legitimate users won't even be delayed while the attack is in progress. That way, the very impractical DoS attack becomes an extremely impractical attack.

Additionally, it makes sense to do more aggressive throttling on admin accounts, since those are the most attractive entry points

PART VII: Distributed Brute Force Attacks

Just as an aside, more advanced attackers will try to circumvent login throttling by 'spreading their activities':

  • Distributing the attempts on a botnet to prevent IP address flagging

  • Rather than picking one user and trying the 50.000 most common passwords (which they can't, because of our throttling), they will pick THE most common password and try it against 50.000 users instead. That way, not only do they get around maximum-attempts measures like CAPTCHAs and login throttling, their chance of success increases as well, since the number 1 most common password is far more likely than number 49.995

  • Spacing the login requests for each user account, say, 30 seconds apart, to sneak under the radar

Here, the best practice would be logging the number of failed logins, system-wide, and using a running average of your site's bad-login frequency as the basis for an upper limit that you then impose on all users.

Too abstract? Let me rephrase:

Say your site has had an average of 120 bad logins per day over the past 3 months. Using that (running average), your system might set the global limit to 3 times that -- ie. 360 failed attempts over a 24 hour period. Then, if the total number of failed attempts across all accounts exceeds that number within one day (or even better, monitor the rate of acceleration and trigger on a calculated threshold), it activates system-wide login throttling - meaning short delays for ALL users (still, with the exception of cookie logins and/or backup CAPTCHA logins).

I also posted a question with more details and a really good discussion of how to avoid tricky pitfals in fending off distributed brute force attacks

PART VIII: Two-Factor Authentication and Authentication Providers

Credentials can be compromised, whether by exploits, passwords being written down and lost, laptops with keys being stolen, or users entering logins into phishing sites. Logins can be further protected with two-factor authentication, which uses out-of-band factors such as single-use codes received from a phone call, SMS message, app, or dongle. Several providers offer two-factor authentication services.

Authentication can be completely delegated to a single-sign-on service, where another provider handles collecting credentials. This pushes the problem to a trusted third party. Google and Twitter both provide standards-based SSO services, while Facebook provides a similar proprietary solution.

MUST-READ LINKS About Web Authentication

  1. OWASP Guide To Authentication / OWASP Authentication Cheat Sheet
  2. Dos and Don’ts of Client Authentication on the Web (very readable MIT research paper)
  3. Wikipedia: HTTP cookie
  4. Personal knowledge questions for fallback authentication: Security questions in the era of Facebook (very readable Berkeley research paper)

How do you make sure email you send programmatically is not automatically marked as spam?

I hate to tell you, but I and others may be using white-list defaults to control our filtering of spam.

This means that all e-mail from an unknown source is automatically spam and diverted into a spam folder. (I don't let my e-mail service delete spam, because I want to always review the arrivals for false positives, something that is pretty easy to do by a quick scan of the folder.)

I even have e-mail from myself go to the spam bucket because (1) I usually don't send e-mail to myself and (2) there are spammers that fake my return address in spam sent to me.

So to get out of the spam designation, I have to consider that your mail might be legitimate (from sender and subject information) and open it first in plaintext (my default for all incoming mail, spam or not) to see if it is legitimate. My spam folder will not use any links in e-mails so I am protected against tricky image links and other misbehavior.

If I want future arrivals from the same source to go to my in box and not be diverted for spam review, I will specify that to my e-mail client. For those organizations that use bulk-mail forwarders and unique sender addresses per mail piece, that's too bad. They never get my approval and always show up in my spam folder, and if I'm busy I will never look at them.

Finally, if an e-mail is not legible in plaintext, even when sent as HTML, I am likely to just delete it unless it is something that I know is of interest to me by virtue of the source and previous valuable experiences.

As you can see, it is ultimately under an users control and there is no automated act that will convince such a system that your mail is legitimate from its structure alone. In this case, you need to play nice, don't do anything that is similar to phishing, and make it easy for users willing to trust your mail to add you to their white list.

Generate list of all possible permutations of a string

I'm not sure why you would want to do this in the first place. The resulting set for any moderately large values of x and y will be huge, and will grow exponentially as x and/or y get bigger.

Lets say your set of possible characters is the 26 lowercase letters of the alphabet, and you ask your application to generate all permutations where length = 5. Assuming you don't run out of memory you'll get 11,881,376 (i.e. 26 to the power of 5) strings back. Bump that length up to 6, and you'll get 308,915,776 strings back. These numbers get painfully large, very quickly.

Here's a solution I put together in Java. You'll need to provide two runtime arguments (corresponding to x and y). Have fun.

public class GeneratePermutations {
    public static void main(String[] args) {
        int lower = Integer.parseInt(args[0]);
        int upper = Integer.parseInt(args[1]);

        if (upper < lower || upper == 0 || lower == 0) {
            System.exit(0);
        }

        for (int length = lower; length <= upper; length++) {
            generate(length, "");
        }
    }

    private static void generate(int length, String partial) {
        if (length <= 0) {
            System.out.println(partial);
        } else {
            for (char c = 'a'; c <= 'z'; c++) {
                generate(length - 1, partial + c);
            }
        }
    }
}

How do you sort a dictionary by value?

The easiest way to get a sorted Dictionary is to use the built in SortedDictionary class:

//Sorts sections according to the key value stored on "sections" unsorted dictionary, which is passed as a constructor argument
System.Collections.Generic.SortedDictionary<int, string> sortedSections = null;
if (sections != null)
{
    sortedSections = new SortedDictionary<int, string>(sections);
}

sortedSections will contain the sorted version of sections

error_log per Virtual Host?

Have you tried adding the php_value error_log '/path/to/php_error_log to your VirtualHost configuration?

Versioning SQL Server database

I would suggest using comparison tools to improvise a version control system for your database. Two good alternatives are xSQL Schema Compare and xSQL Data Compare.

Now, if your goal is to have only the database's schema under version control you can simply use xSQL Schema Compare to generate xSQL Snapshots of the schema and add these files in your version control. Then, to revert or update to a specific version, just compare the current version of the database with the snapshot for the destination version.

Also, if you want to have the data under version control as well, you can use xSQL Data Compare to generate change scripts for you database and add the .sql files in your version control. You could then execute these scripts to revert / update to any version you want. Keep in mind that for the 'revert' functionality you need to generate change scripts that, when executed, will make Version 3 the same as Version 2 and for the 'update' functionality, you need to generate change scripts that do the opposite.

Lastly, with some basic batch programming skills you can automate the whole process by using the command line versions of xSQL Schema Compare and xSQL Data Compare

Disclaimer: I'm affiliated to xSQL.

Embedding Windows Media Player for all browsers

The following works for me in Firefox and Internet Explorer:

<object id="mediaplayer" classid="clsid:22d6f312-b0f6-11d0-94ab-0080c74c7e95" codebase="http://activex.microsoft.com/activex/controls/mplayer/en/nsmp2inf.cab#version=5,1,52,701" standby="loading microsoft windows media player components..." type="application/x-oleobject" width="320" height="310">
<param name="filename" value="./test.wmv">
     <param name="animationatstart" value="true">
     <param name="transparentatstart" value="true">
     <param name="autostart" value="true">
     <param name="showcontrols" value="true">
     <param name="ShowStatusBar" value="true">
     <param name="windowlessvideo" value="true">
     <embed src="./test.wmv" autostart="true" showcontrols="true" showstatusbar="1" bgcolor="white" width="320" height="310">
</object>

Multiple submit buttons in an HTML form

I came across this question when trying to find an answer to basically the same thing, only with ASP.NET controls, when I figured out that the ASP button has a property called UseSubmitBehavior that allows you to set which one does the submitting.

<asp:Button runat="server" ID="SumbitButton" UseSubmitBehavior="False" Text="Submit" />

Just in case someone is looking for the ASP.NET button way to do it.

Difference between Math.Floor() and Math.Truncate()

They are functionally equivalent with positive numbers. The difference is in how they handle negative numbers.

For example:

Math.Floor(2.5) = 2
Math.Truncate(2.5) = 2

Math.Floor(-2.5) = -3
Math.Truncate(-2.5) = -2

MSDN links: - Math.Floor Method - Math.Truncate Method

P.S. Beware of Math.Round it may not be what you expect.

To get the "standard" rounding result use:

float myFloat = 4.5;
Console.WriteLine( Math.Round(myFloat) ); // writes 4
Console.WriteLine( Math.Round(myFloat, 0, MidpointRounding.AwayFromZero) ) //writes 5
Console.WriteLine( myFloat.ToString("F0") ); // writes 5

Determine a user's timezone

You could do it on the client with moment-timezone and send the value to server; sample usage:

> moment.tz.guess()
"America/Asuncion"

Calculate relative time in C#

iPhone Objective-C Version

+ (NSString *)timeAgoString:(NSDate *)date {
    int delta = -(int)[date timeIntervalSinceNow];

    if (delta < 60)
    {
        return delta == 1 ? @"one second ago" : [NSString stringWithFormat:@"%i seconds ago", delta];
    }
    if (delta < 120)
    {
        return @"a minute ago";
    }
    if (delta < 2700)
    {
        return [NSString stringWithFormat:@"%i minutes ago", delta/60];
    }
    if (delta < 5400)
    {
        return @"an hour ago";
    }
    if (delta < 24 * 3600)
    {
        return [NSString stringWithFormat:@"%i hours ago", delta/3600];
    }
    if (delta < 48 * 3600)
    {
        return @"yesterday";
    }
    if (delta < 30 * 24 * 3600)
    {
        return [NSString stringWithFormat:@"%i days ago", delta/(24*3600)];
    }
    if (delta < 12 * 30 * 24 * 3600)
    {
        int months = delta/(30*24*3600);
        return months <= 1 ? @"one month ago" : [NSString stringWithFormat:@"%i months ago", months];
    }
    else
    {
        int years = delta/(12*30*24*3600);
        return years <= 1 ? @"one year ago" : [NSString stringWithFormat:@"%i years ago", years];
    }
}

How do I calculate someone's age based on a DateTime type birthday?

I used ScArcher2's solution for an accurate Year calculation of a persons age but I needed to take it further and calculate their Months and Days along with the Years.

    public static Dictionary<string,int> CurrentAgeInYearsMonthsDays(DateTime? ndtBirthDate, DateTime? ndtReferralDate)
    {
        //----------------------------------------------------------------------
        // Can't determine age if we don't have a dates.
        //----------------------------------------------------------------------
        if (ndtBirthDate == null) return null;
        if (ndtReferralDate == null) return null;

        DateTime dtBirthDate = Convert.ToDateTime(ndtBirthDate);
        DateTime dtReferralDate = Convert.ToDateTime(ndtReferralDate);

        //----------------------------------------------------------------------
        // Create our Variables
        //----------------------------------------------------------------------
        Dictionary<string, int> dYMD = new Dictionary<string,int>();
        int iNowDate, iBirthDate, iYears, iMonths, iDays;
        string sDif = "";

        //----------------------------------------------------------------------
        // Store off current date/time and DOB into local variables
        //---------------------------------------------------------------------- 
        iNowDate = int.Parse(dtReferralDate.ToString("yyyyMMdd"));
        iBirthDate = int.Parse(dtBirthDate.ToString("yyyyMMdd"));

        //----------------------------------------------------------------------
        // Calculate Years
        //----------------------------------------------------------------------
        sDif = (iNowDate - iBirthDate).ToString();
        iYears = int.Parse(sDif.Substring(0, sDif.Length - 4));

        //----------------------------------------------------------------------
        // Store Years in Return Value
        //----------------------------------------------------------------------
        dYMD.Add("Years", iYears);

        //----------------------------------------------------------------------
        // Calculate Months
        //----------------------------------------------------------------------
        if (dtBirthDate.Month > dtReferralDate.Month)
            iMonths = 12 - dtBirthDate.Month + dtReferralDate.Month - 1;
        else
            iMonths = dtBirthDate.Month - dtReferralDate.Month;

        //----------------------------------------------------------------------
        // Store Months in Return Value
        //----------------------------------------------------------------------
        dYMD.Add("Months", iMonths);

        //----------------------------------------------------------------------
        // Calculate Remaining Days
        //----------------------------------------------------------------------
        if (dtBirthDate.Day > dtReferralDate.Day)
            //Logic: Figure out the days in month previous to the current month, or the admitted month.
            //       Subtract the birthday from the total days which will give us how many days the person has lived since their birthdate day the previous month.
            //       then take the referral date and simply add the number of days the person has lived this month.

            //If referral date is january, we need to go back to the following year's December to get the days in that month.
            if (dtReferralDate.Month == 1)
                iDays = DateTime.DaysInMonth(dtReferralDate.Year - 1, 12) - dtBirthDate.Day + dtReferralDate.Day;       
            else
                iDays = DateTime.DaysInMonth(dtReferralDate.Year, dtReferralDate.Month - 1) - dtBirthDate.Day + dtReferralDate.Day;       
        else
            iDays = dtReferralDate.Day - dtBirthDate.Day;             

        //----------------------------------------------------------------------
        // Store Days in Return Value
        //----------------------------------------------------------------------
        dYMD.Add("Days", iDays);

        return dYMD;
}

How do you redirect HTTPS to HTTP?

As far as I'm aware of a simple meta refresh also works without causing errors:

<meta http-equiv="refresh" content="0;URL='http://www.yourdomain.com/path'">

How do you determine the size of a file in C?

You can open the file, go to 0 offset relative from the bottom of the file with

#define SEEKBOTTOM   2

fseek(handle, 0, SEEKBOTTOM)  

the value returned from fseek is the size of the file.

I didn't code in C for a long time, but I think it should work.

Center text output from Graphics.DrawString()

To align a text use the following:

StringFormat sf = new StringFormat();
sf.LineAlignment = StringAlignment.Center;
sf.Alignment = StringAlignment.Center;
e.Graphics.DrawString("My String", this.Font, Brushes.Black, ClientRectangle, sf);

Please note that the text here is aligned in the given bounds. In this sample this is the ClientRectangle.

How do you open a file in C++?

You need to use an ifstream if you just want to read (use an ofstream to write, or an fstream for both).

To open a file in text mode, do the following:

ifstream in("filename.ext", ios_base::in); // the in flag is optional

To open a file in binary mode, you just need to add the "binary" flag.

ifstream in2("filename2.ext", ios_base::in | ios_base::binary ); 

Use the ifstream.read() function to read a block of characters (in binary or text mode). Use the getline() function (it's global) to read an entire line.

Change visibility of ASP.NET label with JavaScript

This is the easiest way I found:

        BtnUpload.Style.Add("display", "none");
        FileUploader.Style.Add("display", "none");
        BtnAccept.Style.Add("display", "inherit");
        BtnClear.Style.Add("display", "inherit");

I have the opposite in the Else, so it handles displaying them as well. This can go in the Page's Load or in a method to refresh the controls on the page.

How do I enable MSDTC on SQL Server?

Do you even need MSDTC? The escalation you're experiencing is often caused by creating multiple connections within a single TransactionScope.

If you do need it then you need to enable it as outlined in the error message. On XP:

  • Go to Administrative Tools -> Component Services
  • Expand Component Services -> Computers ->
  • Right-click -> Properties -> MSDTC tab
  • Hit the Security Configuration button

How to resolve symbolic links in a shell script

Since I've run into this many times over the years, and this time around I needed a pure bash portable version that I could use on OSX and linux, I went ahead and wrote one:

The living version lives here:

https://github.com/keen99/shell-functions/tree/master/resolve_path

but for the sake of SO, here's the current version (I feel it's well tested..but I'm open to feedback!)

Might not be difficult to make it work for plain bourne shell (sh), but I didn't try...I like $FUNCNAME too much. :)

#!/bin/bash

resolve_path() {
    #I'm bash only, please!
    # usage:  resolve_path <a file or directory> 
    # follows symlinks and relative paths, returns a full real path
    #
    local owd="$PWD"
    #echo "$FUNCNAME for $1" >&2
    local opath="$1"
    local npath=""
    local obase=$(basename "$opath")
    local odir=$(dirname "$opath")
    if [[ -L "$opath" ]]
    then
    #it's a link.
    #file or directory, we want to cd into it's dir
        cd $odir
    #then extract where the link points.
        npath=$(readlink "$obase")
        #have to -L BEFORE we -f, because -f includes -L :(
        if [[ -L $npath ]]
         then
        #the link points to another symlink, so go follow that.
            resolve_path "$npath"
            #and finish out early, we're done.
            return $?
            #done
        elif [[ -f $npath ]]
        #the link points to a file.
         then
            #get the dir for the new file
            nbase=$(basename $npath)
            npath=$(dirname $npath)
            cd "$npath"
            ndir=$(pwd -P)
            retval=0
            #done
        elif [[ -d $npath ]]
         then
        #the link points to a directory.
            cd "$npath"
            ndir=$(pwd -P)
            retval=0
            #done
        else
            echo "$FUNCNAME: ERROR: unknown condition inside link!!" >&2
            echo "opath [[ $opath ]]" >&2
            echo "npath [[ $npath ]]" >&2
            return 1
        fi
    else
        if ! [[ -e "$opath" ]]
         then
            echo "$FUNCNAME: $opath: No such file or directory" >&2
            return 1
            #and break early
        elif [[ -d "$opath" ]]
         then 
            cd "$opath"
            ndir=$(pwd -P)
            retval=0
            #done
        elif [[ -f "$opath" ]]
         then
            cd $odir
            ndir=$(pwd -P)
            nbase=$(basename "$opath")
            retval=0
            #done
        else
            echo "$FUNCNAME: ERROR: unknown condition outside link!!" >&2
            echo "opath [[ $opath ]]" >&2
            return 1
        fi
    fi
    #now assemble our output
    echo -n "$ndir"
    if [[ "x${nbase:=}" != "x" ]]
     then
        echo "/$nbase"
    else 
        echo
    fi
    #now return to where we were
    cd "$owd"
    return $retval
}

here's a classic example, thanks to brew:

%% ls -l `which mvn`
lrwxr-xr-x  1 draistrick  502  29 Dec 17 10:50 /usr/local/bin/mvn@ -> ../Cellar/maven/3.2.3/bin/mvn

use this function and it will return the -real- path:

%% cat test.sh
#!/bin/bash
. resolve_path.inc
echo
echo "relative symlinked path:"
which mvn
echo
echo "and the real path:"
resolve_path `which mvn`


%% test.sh

relative symlinked path:
/usr/local/bin/mvn

and the real path:
/usr/local/Cellar/maven/3.2.3/libexec/bin/mvn 

Database, Table and Column Naming Conventions?

our preference:

  1. Should table names be plural?
    Never. The arguments for it being a collection make sense, but you never know what the table is going to contain (0,1 or many items). Plural rules make the naming unnecessarily complicated. 1 House, 2 houses, mouse vs mice, person vs people, and we haven't even looked at any other languages.

    Update person set property = 'value' acts on each person in the table.
    Select * from person where person.name = 'Greg' returns a collection/rowset of person rows.

  2. Should column names be singular?
    Usually, yes, except where you are breaking normalisation rules.

  3. Should I prefix tables or columns?
    Mostly a platform preference. We prefer to prefix columns with the table name. We don't prefix tables, but we do prefix views (v_) and stored_procedures (sp_ or f_ (function)). That helps people who want to try to upday v_person.age which is actually a calculated field in a view (which can't be UPDATEd anyway).

    It is also a great way to avoid keyword collision (delivery.from breaks, but delivery_from does not).

    It does make the code more verbose, but often aids in readability.

    bob = new person()
    bob.person_name = 'Bob'
    bob.person_dob = '1958-12-21'
    ... is very readable and explicit. This can get out of hand though:

    customer.customer_customer_type_id

    indicates a relationship between customer and the customer_type table, indicates the primary key on the customer_type table (customer_type_id) and if you ever see 'customer_customer_type_id' whilst debugging a query, you know instantly where it is from (customer table).

    or where you have a M-M relationship between customer_type and customer_category (only certain types are available to certain categories)

    customer_category_customer_type_id

    ... is a little (!) on the long side.

  4. Should I use any case in naming items? Yes - lower case :), with underscores. These are very readable and cross platform. Together with 3 above it also makes sense.

    Most of these are preferences though. - As long as you are consistent, it should be predictable for anyone that has to read it.

How do I remove duplicate items from an array in Perl?

Try this, seems the uniq function needs a sorted list to work properly.

use strict;

# Helper function to remove duplicates in a list.
sub uniq {
  my %seen;
  grep !$seen{$_}++, @_;
}

my @teststrings = ("one", "two", "three", "one");

my @filtered = uniq @teststrings;
print "uniq: @filtered\n";
my @sorted = sort @teststrings;
print "sort: @sorted\n";
my @sortedfiltered = uniq sort @teststrings;
print "uniq sort : @sortedfiltered\n";

Are the shift operators (<<, >>) arithmetic or logical in C?

will typically use logical shifts on unsigned variables and for left-shifts on signed variables. The arithmetic right shift is the truly important one because it will sign extend the variable.

will will use this when applicable, as other compilers are likely to do.

Best Practices for securing a REST API / web service

It's been a while but the question is still relevant, though the answer might have changed a bit.

An API Gateway would be a flexible and highly configurable solution. I tested and used KONG quite a bit and really liked what I saw. KONG provides an admin REST API of its own which you can use to manage users.

Express-gateway.io is more recent and is also an API Gateway.

Performing a Stress Test on Web Application?

We recently started using Gatling for load testing. I would highly recommend to try out this tool for load testing. We had used SOASTA and JMETER in past. Our main reason to consider Gatling is following:

  • Recorder to record the scenario
  • Using Akka and Netty which gives better performance compare to Jmeter Threading model
  • DSL Scala which is very much maintainable compare to Jmeter XML
  • Easy to write the tests, don't scare if it's scala.
  • Reporting

Let me give you simple example to write the code using Gatling Code:

// your code starts here  
val scn = scenario("Scenario")  
     .exec(http("Page")
     .get("http://example.com")) 
// injecting 100 user enter code here's on above scenario.   
setUp(scn.inject(atOnceUsers(100)))       

However you can make it as complicated as possible. One of the feature which stand out for Gatling is reporting which is very detail.

Here are some links:
Gatling
Gatling Tutorial

I recently gave talk on it, you can go through the talk here:
https://docs.google.com/viewer?url=http%3A%2F%2Ffiles.meetup.com%2F3872152%2FExploring-Load-Testing-with-Gatling.pdf

How to autosize a textarea using Prototype?

Like the answer of @memical.

However I found some improvements. You can use the jQuery height() function. But be aware of padding-top and padding-bottom pixels. Otherwise your textarea will grow too fast.

$(document).ready(function() {
  $textarea = $("#my-textarea");

  // There is some diff between scrollheight and height:
  //    padding-top and padding-bottom
  var diff = $textarea.prop("scrollHeight") - $textarea.height();
  $textarea.live("keyup", function() {
    var height = $textarea.prop("scrollHeight") - diff;
    $textarea.height(height);
  });
});

PDF Editing in PHP?

There is a free and easy to use PDF class to create PDF documents. It's called FPDF. In combination with FPDI (http://www.setasign.de/products/pdf-php-solutions/fpdi) it is even possible to edit PDF documents. The following code shows how to use FPDF and FPDI to fill an existing gift coupon with the user data.

require_once('fpdf.php'); 
require_once('fpdi.php'); 
$pdf = new FPDI();

$pdf->AddPage(); 

$pdf->setSourceFile('gift_coupon.pdf'); 
// import page 1 
$tplIdx = $this->pdf->importPage(1); 
//use the imported page and place it at point 0,0; calculate width and height
//automaticallay and ajust the page size to the size of the imported page 
$this->pdf->useTemplate($tplIdx, 0, 0, 0, 0, true); 

// now write some text above the imported page 
$this->pdf->SetFont('Arial', '', '13'); 
$this->pdf->SetTextColor(0,0,0);
//set position in pdf document
$this->pdf->SetXY(20, 20);
//first parameter defines the line height
$this->pdf->Write(0, 'gift code');
//force the browser to download the output
$this->pdf->Output('gift_coupon_generated.pdf', 'D');

What is Turing Complete?

In practical language terms familiar to most programmers, the usual way to detect Turing completeness is if the language allows or allows the simulation of nested unbounded while statements (as opposed to Pascal-style for statements, with fixed upper bounds).

What is the difference between String and string in C#?

String (capital S) is a class in the .NET framework in the System namespace. The fully qualified name is System.String. Whereas, the lower case string is an alias of System.String.

string str1= "Hello";
String str2 = "World!";
            
Console.WriteLine(str1.GetType().FullName); // System.String
Console.WriteLine(str2.GetType().FullName); // System.String

How to show a GUI message box from a bash script in linux?

The zenity application appears to be what you are looking for.

To take input from zenity, you can specify a variable and have the output of zenity --entry saved to it. It looks something like this:

my_variable=$(zenity --entry)

If you look at the value in my_variable now, it will be whatever was typed in the zenity pop up entry dialog.

If you want to give some sort of prompt as to what the user (or you) should enter in the dialog, add the --text switch with the label that you want. It looks something like this:

my_variable=$(zenity --entry --text="What's my variable:")

Zenity has lot of other nice options that are for specific tasks, so you might want to check those out as well with zenity --help. One example is the --calendar option that let's you select a date from a graphical calendar.

my_date=$(zenity --calendar)

Which gives a nicely formatted date based on what the user clicked on:

echo ${my_date}

gives:

08/05/2009

There are also options for slider selectors, errors, lists and so on.

Hope this helps.

Graph visualization library in JavaScript

I've just put together what you may be looking for: http://www.graphdracula.net

It's JavaScript with directed graph layouting, SVG and you can even drag the nodes around. Still needs some tweaking, but is totally usable. You create nodes and edges easily with JavaScript code like this:

var g = new Graph();
g.addEdge("strawberry", "cherry");
g.addEdge("cherry", "apple");
g.addEdge("id34", "cherry");

I used the previously mentioned Raphael JS library (the graffle example) plus some code for a force based graph layout algorithm I found on the net (everything open source, MIT license). If you have any remarks or need a certain feature, I may implement it, just ask!


You may want to have a look at other projects, too! Below are two meta-comparisons:

  • SocialCompare has an extensive list of libraries, and the "Node / edge graph" line will filter for graph visualization ones.

  • DataVisualization.ch has evaluated many libraries, including node/graph ones. Unfortunately there's no direct link so you'll have to filter for "graph":Selection DataVisualization.ch

Here's a list of similar projects (some have been already mentioned here):

Pure JavaScript Libraries

  • vis.js supports many types of network/edge graphs, plus timelines and 2D/3D charts. Auto-layout, auto-clustering, springy physics engine, mobile-friendly, keyboard navigation, hierarchical layout, animation etc. MIT licensed and developed by a Dutch firm specializing in research on self-organizing networks.

  • Cytoscape.js - interactive graph analysis and visualization with mobile support, following jQuery conventions. Funded via NIH grants and developed by by @maxkfranz (see his answer below) with help from several universities and other organizations.

  • The JavaScript InfoVis Toolkit - Jit, an interactive, multi-purpose graph drawing and layout framework. See for example the Hyperbolic Tree. Built by Twitter dataviz architect Nicolas Garcia Belmonte and bought by Sencha in 2010.

  • D3.js Powerful multi-purpose JS visualization library, the successor of Protovis. See the force-directed graph example, and other graph examples in the gallery.

  • Plotly's JS visualization library uses D3.js with JS, Python, R, and MATLAB bindings. See a nexworkx example in IPython here, human interaction example here, and JS Embed API.

  • sigma.js Lightweight but powerful library for drawing graphs

  • jsPlumb jQuery plug-in for creating interactive connected graphs

  • Springy - a force-directed graph layout algorithm

  • Processing.js Javascript port of the Processing library by John Resig

  • JS Graph It - drag'n'drop boxes connected by straight lines. Minimal auto-layout of the lines.

  • RaphaelJS's Graffle - interactive graph example of a generic multi-purpose vector drawing library. RaphaelJS can't layout nodes automatically; you'll need another library for that.

  • JointJS Core - David Durman's MPL-licensed open source diagramming library. It can be used to create either static diagrams or fully interactive diagramming tools and application builders. Works in browsers supporting SVG. Layout algorithms not-included in the core package

  • mxGraph Previously commercial HTML 5 diagramming library, now available under Apache v2.0. mxGraph is the base library used in draw.io.

Commercial libraries

Abandoned libraries

  • Cytoscape Web Embeddable JS Network viewer (no new features planned; succeeded by Cytoscape.js)

  • Canviz JS renderer for Graphviz graphs. Abandoned in Sep 2013.

  • arbor.js Sophisticated graphing with nice physics and eye-candy. Abandoned in May 2012. Several semi-maintained forks exist.

  • jssvggraph "The simplest possible force directed graph layout algorithm implemented as a Javascript library that uses SVG objects". Abandoned in 2012.

  • jsdot Client side graph drawing application. Abandoned in 2011.

  • Protovis Graphical Toolkit for Visualization (JavaScript). Replaced by d3.

  • Moo Wheel Interactive JS representation for connections and relations (2008)

  • JSViz 2007-era graph visualization script

  • dagre Graph layout for JavaScript

Non-Javascript Libraries

How to create a SQL Server function to "join" multiple rows from a subquery into a single delimited field?

Try this query

SELECT v.VehicleId, v.Name, ll.LocationList
FROM Vehicles v 
LEFT JOIN 
    (SELECT 
     DISTINCT
        VehicleId,
        REPLACE(
            REPLACE(
                REPLACE(
                    (
                        SELECT City as c 
                        FROM Locations x 
                        WHERE x.VehicleID = l.VehicleID FOR XML PATH('')
                    ),    
                    '</c><c>',', '
                 ),
             '<c>',''
            ),
        '</c>', ''
        ) AS LocationList
    FROM Locations l
) ll ON ll.VehicleId = v.VehicleId

How should I load files into my Java application?

What are you loading the files for - configuration or data (like an input file) or as a resource?

  • If as a resource, follow the suggestion and example given by Will and Justin
  • If configuration, then you can use a ResourceBundle or Spring (if your configuration is more complex).
  • If you need to read a file in order to process the data inside, this code snippet may help BufferedReader file = new BufferedReader(new FileReader(filename)) and then read each line of the file using file.readLine(); Don't forget to close the file.

C# loop - break vs. continue

By example

foreach(var i in Enumerable.Range(1,3))
{
    Console.WriteLine(i);
}

Prints 1, 2, 3 (on separate lines).

Add a break condition at i = 2

foreach(var i in Enumerable.Range(1,3))
{
    if (i == 2)
        break;

    Console.WriteLine(i);
}

Now the loop prints 1 and stops.

Replace the break with a continue.

foreach(var i in Enumerable.Range(1,3))
{
    if (i == 2)
        continue;

    Console.WriteLine(i);
}

Now to loop prints 1 and 3 (skipping 2).

Thus, break stops the loop, whereas continue skips to the next iteration.

Log4Net configuring log level

If you would like to perform it dynamically try this:

using System;
using System.Collections.Generic;
using System.Text;
using log4net;
using log4net.Config;
using NUnit.Framework;

namespace ExampleConsoleApplication
{
  enum DebugLevel : int
  { 
    Fatal_Msgs = 0 , 
    Fatal_Error_Msgs = 1 , 
    Fatal_Error_Warn_Msgs = 2 , 
    Fatal_Error_Warn_Info_Msgs = 3 ,
    Fatal_Error_Warn_Info_Debug_Msgs = 4 
  }

  class TestClass
  {
    private static readonly ILog logger = LogManager.GetLogger(typeof(TestClass));

    static void Main ( string[] args )
    {
      TestClass objTestClass = new TestClass ();

      Console.WriteLine ( " START " );

      int shouldLog = 4; //CHANGE THIS FROM 0 TO 4 integer to check the functionality of the example
      //0 -- prints only FATAL messages 
      //1 -- prints FATAL and ERROR messages 
      //2 -- prints FATAL , ERROR and WARN messages 
      //3 -- prints FATAL  , ERROR , WARN and INFO messages 
      //4 -- prints FATAL  , ERROR , WARN , INFO and DEBUG messages 

      string srtLogLevel = String.Empty; 
      switch (shouldLog)
      {
        case (int)DebugLevel.Fatal_Msgs :
          srtLogLevel = "FATAL";
          break;
        case (int)DebugLevel.Fatal_Error_Msgs:
          srtLogLevel = "ERROR";
          break;
        case (int)DebugLevel.Fatal_Error_Warn_Msgs :
          srtLogLevel = "WARN";
          break;
        case (int)DebugLevel.Fatal_Error_Warn_Info_Msgs :
          srtLogLevel = "INFO"; 
          break;
        case (int)DebugLevel.Fatal_Error_Warn_Info_Debug_Msgs :
          srtLogLevel = "DEBUG" ;
          break ;
        default:
          srtLogLevel = "FATAL";
          break;
      }

      objTestClass.SetLogingLevel ( srtLogLevel );


      objTestClass.LogSomething ();


      Console.WriteLine ( " END HIT A KEY TO EXIT " );
      Console.ReadLine ();
    } //eof method 

    /// <summary>
    /// Activates debug level 
    /// </summary>
    /// <sourceurl>http://geekswithblogs.net/rakker/archive/2007/08/22/114900.aspx</sourceurl>
    private void SetLogingLevel ( string strLogLevel )
    {
     string strChecker = "WARN_INFO_DEBUG_ERROR_FATAL" ;

      if (String.IsNullOrEmpty ( strLogLevel ) == true || strChecker.Contains ( strLogLevel ) == false)
        throw new Exception ( " The strLogLevel should be set to WARN , INFO , DEBUG ," );



      log4net.Repository.ILoggerRepository[] repositories = log4net.LogManager.GetAllRepositories ();

      //Configure all loggers to be at the debug level.
      foreach (log4net.Repository.ILoggerRepository repository in repositories)
      {
        repository.Threshold = repository.LevelMap[ strLogLevel ];
        log4net.Repository.Hierarchy.Hierarchy hier = (log4net.Repository.Hierarchy.Hierarchy)repository;
        log4net.Core.ILogger[] loggers = hier.GetCurrentLoggers ();
        foreach (log4net.Core.ILogger logger in loggers)
        {
          ( (log4net.Repository.Hierarchy.Logger)logger ).Level = hier.LevelMap[ strLogLevel ];
        }
      }

      //Configure the root logger.
      log4net.Repository.Hierarchy.Hierarchy h = (log4net.Repository.Hierarchy.Hierarchy)log4net.LogManager.GetRepository ();
      log4net.Repository.Hierarchy.Logger rootLogger = h.Root;
      rootLogger.Level = h.LevelMap[ strLogLevel ];
    }

    private void LogSomething ()
    {
      #region LoggerUsage
      DOMConfigurator.Configure (); //tis configures the logger 
      logger.Debug ( "Here is a debug log." );
      logger.Info ( "... and an Info log." );
      logger.Warn ( "... and a warning." );
      logger.Error ( "... and an error." );
      logger.Fatal ( "... and a fatal error." );
      #endregion LoggerUsage

    }
  } //eof class 
} //eof namespace 

The app config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <configSections>
        <section name="log4net"
                 type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
    </configSections>
    <log4net>
        <appender name="LogFileAppender" type="log4net.Appender.FileAppender">
            <param name="File" value="LogTest2.txt" />
            <param name="AppendToFile" value="true" />
            <layout type="log4net.Layout.PatternLayout">
                <param name="Header" value="[Header] \r\n" />
                <param name="Footer" value="[Footer] \r\n" />
                <param name="ConversionPattern" value="%d [%t] %-5p %c %m%n" />
            </layout>
        </appender>

        <appender name="ColoredConsoleAppender" type="log4net.Appender.ColoredConsoleAppender">
            <mapping>
                <level value="ERROR" />
                <foreColor value="White" />
                <backColor value="Red, HighIntensity" />
            </mapping>
            <layout type="log4net.Layout.PatternLayout">
                <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
            </layout>
        </appender>


        <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">
            <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.2.10.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
            <connectionString value="data source=ysg;initial catalog=DBGA_DEV;integrated security=true;persist security info=True;" />
            <commandText value="INSERT INTO [DBGA_DEV].[ga].[tb_Data_Log] ([Date],[Thread],[Level],[Logger],[Message]) VALUES (@log_date, @thread, @log_level, @logger, @message)" />

            <parameter>
                <parameterName value="@log_date" />
                <dbType value="DateTime" />
                <layout type="log4net.Layout.PatternLayout" value="%date{yyyy'-'MM'-'dd HH':'mm':'ss'.'fff}" />
            </parameter>
            <parameter>
                <parameterName value="@thread" />
                <dbType value="String" />
                <size value="255" />
                <layout type="log4net.Layout.PatternLayout" value="%thread" />
            </parameter>
            <parameter>
                <parameterName value="@log_level" />
                <dbType value="String" />
                <size value="50" />
                <layout type="log4net.Layout.PatternLayout" value="%level" />
            </parameter>
            <parameter>
                <parameterName value="@logger" />
                <dbType value="String" />
                <size value="255" />
                <layout type="log4net.Layout.PatternLayout" value="%logger" />
            </parameter>
            <parameter>
                <parameterName value="@message" />
                <dbType value="String" />
                <size value="4000" />
                <layout type="log4net.Layout.PatternLayout" value="%messag2e" />
            </parameter>
        </appender>
        <root>
            <level value="INFO" />
            <appender-ref ref="LogFileAppender" />
            <appender-ref ref="AdoNetAppender" />
            <appender-ref ref="ColoredConsoleAppender" />
        </root>
    </log4net>
</configuration>

The references in the csproj file:

<Reference Include="log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821, processorArchitecture=MSIL">
  <SpecificVersion>False</SpecificVersion>
  <HintPath>..\..\..\Log4Net\log4net-1.2.10\bin\net\2.0\release\log4net.dll</HintPath>
</Reference>
<Reference Include="nunit.framework, Version=2.4.8.0, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL" />

Getting the text from a drop-down box

This works i tried it my self i thought i post it here in case someone need it...

document.getElementById("newSkill").options[document.getElementById('newSkill').selectedIndex].text;

SQL Server Escape an Underscore

I had a similar issue using like pattern '%_%' did not work - as the question indicates :-)

Using '%\_%' did not work either as this first \ is interpreted "before the like".

Using '%\\_%' works. The \\ (double backslash) is first converted to single \ (backslash) and then used in the like pattern.

IsNothing versus Is Nothing

Is Nothing requires an object that has been assigned to the value Nothing. IsNothing() can take any variable that has not been initialized, including of numeric type. This is useful for example when testing if an optional parameter has been passed.

Tab Escape Character?

For someone who needs quick reference of C# Escape Sequences that can be used in string literals:

\t     Horizontal tab (ASCII code value: 9)

\n     Line feed (ASCII code value: 10)

\r     Carriage return (ASCII code value: 13)

\'     Single quotation mark

\"     Double quotation mark

\\     Backslash

\?     Literal question mark

\x12     ASCII character in hexadecimal notation (e.g. for 0x12)

\x1234     Unicode character in hexadecimal notation (e.g. for 0x1234)

It's worth mentioning that these (in most cases) are universal codes. So \t is 9 and \n is 10 char value on Windows and Linux. But newline sequence is not universal. On Windows it's \n\r and on Linux it's just \n. That's why it's best to use Environment.Newline which gets adjusted to current OS settings. With .Net Core it gets really important.

Equivalent VB keyword for 'break'

Exit [construct], and intelisense will tell you which one(s) are valid in a particular place.

The imported project "C:\Microsoft.CSharp.targets" was not found

This error can also occur when opening a Silverlight project that was built in SL 4, while you have SL 5 installed.

Here is an example error message: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\Silverlight\v4.0\Microsoft.Silverlight.CSharp.targets" was not found.

Note the v4.0.

To resolve, edit the project and find:

<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

And change it to v5.0.

Then reload project and it will open (unless you do not have SL 5 installed).

x86 Assembly on a Mac

Recently I wanted to learn how to compile Intel x86 on Mac OS X:

For nasm:

-o hello.tmp - outfile
-f macho - specify format
Linux - elf or elf64
Mac OSX - macho

For ld:

-arch i386 - specify architecture (32 bit assembly)
-macosx_version_min 10.6 (Mac OSX - complains about default specification)
-no_pie (Mac OSX - removes ld warning)
-e main - specify main symbol name (Mac OSX - default is start)
-o hello.o - outfile

For Shell:

./hello.o - execution

One-liner:

nasm -o hello.tmp -f macho hello.s && ld -arch i386 -macosx_version_min 10.6 -no_pie -e _main -o hello.o hello.tmp && ./hello.o

Let me know if this helps!

I wrote how to do it on my blog here:

http://blog.burrowsapps.com/2013/07/how-to-compile-helloworld-in-intel-x86.html

For a more verbose explanation, I explained on my Github here:

https://github.com/jaredsburrows/Assembly

How can I undo git reset --hard HEAD~1?

My problem is almost similar. I have uncommitted files before I enter git reset --hard.

Thankfully. I managed to skip all these resources. After I noticed that I can just undo (ctrl-z). I just want to add this to all of the answers above.

Note. Its not possible to ctrl-z unopened files.

Python, Unicode, and the Windows console

Just enter this code in command line before executing python script:

chcp 65001 & set PYTHONIOENCODING=utf-8

Length of a JavaScript object

The solution work for many cases and cross browser:

Code

var getTotal = function(collection) {

    var length = collection['length'];
    var isArrayObject =  typeof length == 'number' && length >= 0 && length <= Math.pow(2,53) - 1; // Number.MAX_SAFE_INTEGER

    if(isArrayObject) {
        return collection['length'];
    }

    i= 0;
    for(var key in collection) {
        if (collection.hasOwnProperty(key)) {
            i++;
        }
    }

    return i;
};

Data Examples:

// case 1
var a = new Object();
a["firstname"] = "Gareth";
a["lastname"] = "Simpson";
a["age"] = 21;

//case 2
var b = [1,2,3];

// case 3
var c = {};
c[0] = 1;
c.two = 2;

Usage

getLength(a); // 3
getLength(b); // 3
getLength(c); // 2

Accessing post variables using Java Servlets

For getting all post parameters there is Map which contains request param name as key and param value as key.

Map params = servReq.getParameterMap();

And to get parameters with known name normal

String userId=servReq.getParameter("user_id");

Setting a div's height in HTML with CSS

I had the same problem on my site (shameless plug).

I had the nav section "float: right" and the main body of the page has a background image about 250px across aligned to the right and "repeat-y". I then added something with "clear: both" to it. Here is the W3Schools and the CSS clear property.

I placed the clear at the bottom of the "page" classed div. My page source looks something like this.

body
 -> header (big blue banner)
 -> headerNav (green bar at the top)
 -> breadcrumbs (invisible at the moment)
 -> page
     -> navigation (floats to the right)
     -> content (main content)
         -> clear (the quote at the bottom)
         -> footerNav (the green bar at the bottom)
     -> clear (empty but still does something)
 -> footer (blue thing at the bottom)

I hope that helps :)

Using ConfigurationManager to load config from an arbitrary location

Try this:

System.Configuration.ConfigurationFileMap fileMap = new ConfigurationFileMap(strConfigPath); //Path to your config file
System.Configuration.Configuration configuration = System.Configuration.ConfigurationManager.OpenMappedMachineConfiguration(fileMap);

Learning Regular Expressions

The most important part is the concepts. Once you understand how the building blocks work, differences in syntax amount to little more than mild dialects. A layer on top of your regular expression engine's syntax is the syntax of the programming language you're using. Languages such as Perl remove most of this complication, but you'll have to keep in mind other considerations if you're using regular expressions in a C program.

If you think of regular expressions as building blocks that you can mix and match as you please, it helps you learn how to write and debug your own patterns but also how to understand patterns written by others.

Start simple

Conceptually, the simplest regular expressions are literal characters. The pattern N matches the character 'N'.

Regular expressions next to each other match sequences. For example, the pattern Nick matches the sequence 'N' followed by 'i' followed by 'c' followed by 'k'.

If you've ever used grep on Unix—even if only to search for ordinary looking strings—you've already been using regular expressions! (The re in grep refers to regular expressions.)

Order from the menu

Adding just a little complexity, you can match either 'Nick' or 'nick' with the pattern [Nn]ick. The part in square brackets is a character class, which means it matches exactly one of the enclosed characters. You can also use ranges in character classes, so [a-c] matches either 'a' or 'b' or 'c'.

The pattern . is special: rather than matching a literal dot only, it matches any character. It's the same conceptually as the really big character class [-.?+%$A-Za-z0-9...].

Think of character classes as menus: pick just one.

Helpful shortcuts

Using . can save you lots of typing, and there are other shortcuts for common patterns. Say you want to match a digit: one way to write that is [0-9]. Digits are a frequent match target, so you could instead use the shortcut \d. Others are \s (whitespace) and \w (word characters: alphanumerics or underscore).

The uppercased variants are their complements, so \S matches any non-whitespace character, for example.

Once is not enough

From there, you can repeat parts of your pattern with quantifiers. For example, the pattern ab?c matches 'abc' or 'ac' because the ? quantifier makes the subpattern it modifies optional. Other quantifiers are

  • * (zero or more times)
  • + (one or more times)
  • {n} (exactly n times)
  • {n,} (at least n times)
  • {n,m} (at least n times but no more than m times)

Putting some of these blocks together, the pattern [Nn]*ick matches all of

  • ick
  • Nick
  • nick
  • Nnick
  • nNick
  • nnick
  • (and so on)

The first match demonstrates an important lesson: * always succeeds! Any pattern can match zero times.

A few other useful examples:

  • [0-9]+ (and its equivalent \d+) matches any non-negative integer
  • \d{4}-\d{2}-\d{2} matches dates formatted like 2019-01-01

Grouping

A quantifier modifies the pattern to its immediate left. You might expect 0abc+0 to match '0abc0', '0abcabc0', and so forth, but the pattern immediately to the left of the plus quantifier is c. This means 0abc+0 matches '0abc0', '0abcc0', '0abccc0', and so on.

To match one or more sequences of 'abc' with zeros on the ends, use 0(abc)+0. The parentheses denote a subpattern that can be quantified as a unit. It's also common for regular expression engines to save or "capture" the portion of the input text that matches a parenthesized group. Extracting bits this way is much more flexible and less error-prone than counting indices and substr.

Alternation

Earlier, we saw one way to match either 'Nick' or 'nick'. Another is with alternation as in Nick|nick. Remember that alternation includes everything to its left and everything to its right. Use grouping parentheses to limit the scope of |, e.g., (Nick|nick).

For another example, you could equivalently write [a-c] as a|b|c, but this is likely to be suboptimal because many implementations assume alternatives will have lengths greater than 1.

Escaping

Although some characters match themselves, others have special meanings. The pattern \d+ doesn't match backslash followed by lowercase D followed by a plus sign: to get that, we'd use \\d\+. A backslash removes the special meaning from the following character.

Greediness

Regular expression quantifiers are greedy. This means they match as much text as they possibly can while allowing the entire pattern to match successfully.

For example, say the input is

"Hello," she said, "How are you?"

You might expect ".+" to match only 'Hello,' and will then be surprised when you see that it matched from 'Hello' all the way through 'you?'.

To switch from greedy to what you might think of as cautious, add an extra ? to the quantifier. Now you understand how \((.+?)\), the example from your question works. It matches the sequence of a literal left-parenthesis, followed by one or more characters, and terminated by a right-parenthesis.

If your input is '(123) (456)', then the first capture will be '123'. Non-greedy quantifiers want to allow the rest of the pattern to start matching as soon as possible.

(As to your confusion, I don't know of any regular-expression dialect where ((.+?)) would do the same thing. I suspect something got lost in transmission somewhere along the way.)

Anchors

Use the special pattern ^ to match only at the beginning of your input and $ to match only at the end. Making "bookends" with your patterns where you say, "I know what's at the front and back, but give me everything between" is a useful technique.

Say you want to match comments of the form

-- This is a comment --

you'd write ^--\s+(.+)\s+--$.

Build your own

Regular expressions are recursive, so now that you understand these basic rules, you can combine them however you like.

Tools for writing and debugging regexes:

Books

Free resources

Footnote

†: The statement above that . matches any character is a simplification for pedagogical purposes that is not strictly true. Dot matches any character except newline, "\n", but in practice you rarely expect a pattern such as .+ to cross a newline boundary. Perl regexes have a /s switch and Java Pattern.DOTALL, for example, to make . match any character at all. For languages that don't have such a feature, you can use something like [\s\S] to match "any whitespace or any non-whitespace", in other words anything.

Recommended Fonts for Programming?

I use Terminuse in almost everything (Eclipse, putty and other terminals): http://fractal.csie.org/~eric/wiki/Terminus_font

I must say that I don't get it why most people use small fonts like 9pt, do you have 14" monitors or what?

For me the best way is to use font size that makes my monitor display at most one 30-40 line method, this way I need to create smaller methods :)

SQL Case Expression Syntax?

Here you can find a complete guide for MySQL case statements in SQL.

CASE
     WHEN some_condition THEN return_some_value
     ELSE return_some_other_value
END

How to easily consume a web service from PHP

This article explains how you can use PHP SoapClient to call a api web service.

How do I update Ruby Gems from behind a Proxy (ISA-NTLM)

This totally worked:

gem install --http-proxy http://COMPANY.PROXY.ADDRESS $gem_name

Drop all tables whose names begin with a certain string

In case of temporary tables, you might want to try

SELECT 'DROP TABLE "' + t.name + '"' 
FROM tempdb.sys.tables t
WHERE t.name LIKE '[prefix]%'

How do I retrieve my MySQL username and password?

Login MySql from windows cmd using existing user:

mysql -u username -p
Enter password:****

Then run the following command:

mysql> SELECT * FROM mysql.user;

After that copy encrypted md5 password for corresponding user and there are several online password decrypted application available in web. Using this decrypt password and use this for login in next time. or update user password using flowing command:

mysql> UPDATE mysql.user SET Password=PASSWORD('[password]') WHERE User='[username]';

Then login using the new password and user.

How to include PHP files that require an absolute path?

require(str_repeat('../',(substr_count(getenv('SCRIPT_URL'),'/')-1))."/path/to/file.php");

I use this line of code. It goes back to the "top" of the site tree, then goes to the file desired.

For example, let's say i have this file tree:

domain.com/aaa/index.php
domain.com/bbb/ccc/ddd/index.php
domain.com/_resources/functions.php

I can include the functions.php file from wherever i am, just by copy pasting

require(str_repeat('../',(substr_count(getenv('SCRIPT_URL'),'/')-1))."/_resources/functions.php");

If you need to use this code many times, you may create a function that returns the str_repeat('../',(substr_count(getenv('SCRIPT_URL'),'/')-1)) part. Then just insert this function in the first file you include. I have an "initialize.php" file that i include at the very top of each php page and which contains this function. The next time i have to include files, i in fact just use the function (named path_back):

require(path_back()."/_resources/another_php_file.php");

Accessing a Dictionary.Keys Key through a numeric index

I don't know if this would work because I'm pretty sure that the keys aren't stored in the order they are added, but you could cast the KeysCollection to a List and then get the last key in the list... but it would be worth having a look.

The only other thing I can think of is to store the keys in a lookup list and add the keys to the list before you add them to the dictionary... it's not pretty tho.

What Are Some Good .NET Profilers?

Don't forget the awesome scitech .net memory profiler

It's great for tracking down why your .net app is running out of memory.

IllegalArgumentException or NullPointerException for a null parameter?

Actually, the question of throwing IllegalArgumentException or NullPointerException is in my humble view only a "holy war" for a minority with an incomlete understanding of exception handling in Java. In general, the rules are simple, and as follows:

  • argument constraint violations must be indicated as fast as possible (-> fast fail), in order to avoid illegal states which are much harder to debug
  • in case of an invalid null pointer for whatever reason, throw NullPointerException
  • in case of an illegal array/collection index, throw ArrayIndexOutOfBounds
  • in case of a negative array/collection size, throw NegativeArraySizeException
  • in case of an illegal argument that is not covered by the above, and for which you don't have another more specific exception type, throw IllegalArgumentException as a wastebasket
  • on the other hand, in case of a constraint violation WITHIN A FIELD that could not be avoided by fast fail for some valid reason, catch and rethrow as IllegalStateException or a more specific checked exception. Never let pass the original NullPointerException, ArrayIndexOutOfBounds, etc in this case!

There are at least three very good reasons against the case of mapping all kinds of argument constraint violations to IllegalArgumentException, with the third probably being so severe as to mark the practice bad style:

(1) A programmer cannot a safely assume that all cases of argument constraint violations result in IllegalArgumentException, because the large majority of standard classes use this exception rather as a wastebasket if there is no more specific kind of exception available. Trying to map all cases of argument constraint violations to IllegalArgumentException in your API only leads to programmer frustration using your classes, as the standard libraries mostly follow different rules that violate yours, and most of your API users will use them as well!

(2) Mapping the exceptions actually results in a different kind of anomaly, caused by single inheritance: All Java exceptions are classes, and therefore support single inheritance only. Therefore, there is no way to create an exception that is truly say both a NullPointerException and an IllegalArgumentException, as subclasses can only inherit from one or the other. Throwing an IllegalArgumentException in case of a null argument therefore makes it harder for API users to distinguish between problems whenever a program tries to programmatically correct the problem, for example by feeding default values into a call repeat!

(3) Mapping actually creates the danger of bug masking: In order to map argument constraint violations into IllegalArgumentException, you'll need to code an outer try-catch within every method that has any constrained arguments. However, simply catching RuntimeException in this catch block is out of the question, because that risks mapping documented RuntimeExceptions thrown by libery methods used within yours into IllegalArgumentException, even if they are no caused by argument constraint violations. So you need to be very specific, but even that effort doesn't protect you from the case that you accidentally map an undocumented runtime exception of another API (i.e. a bug) into an IllegalArgumentException of your API. Even the most careful mapping therefore risks masking programming errors of other library makers as argument constraint violations of your method's users, which is simply hillareous behavior!

With the standard practice on the other hand, the rules stay simple, and exception causes stay unmasked and specific. For the method caller, the rules are easy as well: - if you encounter a documented runtime exception of any kind because you passed an illegal value, either repeat the call with a default (for this specific exceptions are neccessary), or correct your code - if on the other hand you enccounter a runtime exception that is not documented to happen for a given set of arguments, file a bug report to the method's makers to ensure that either their code or their documentation is fixed.

Storing Images in DB - Yea or Nay?

In a previous project i stored images on the filesystem, and that caused a lot of headaches with backups, replication, and the filesystem getting out of sync with the database.

In my latest project i'm storing images in the database, and caching them on the filesystem, and it works really well. I've had no problems so far.

Call ASP.NET function from JavaScript?

The Microsoft AJAX library will accomplish this. You could also create your own solution that involves using AJAX to call your own aspx (as basically) script files to run .NET functions.

This is the library called AjaxPro which was written an MVP named Michael Schwarz. This was library was not written by Microsoft.

I have used AjaxPro extensively, and it is a very nice library, that I would recommend for simple callbacks to the server. It does function well with the Microsoft version of Ajax with no issues. However, I would note, with how easy Microsoft has made Ajax, I would only use it if really necessary. It takes a lot of JavaScript to do some really complicated functionality that you get from Microsoft by just dropping it into an update panel.

Create a new Ruby on Rails application using MySQL instead of SQLite

rails new <project_name> -d mysql

OR

rails new projectname

Changes in config/database.yml

development:
  adapter: mysql2
  database: db_name_name
  username: root
  password:
  host: localhost
  socket: /tmp/mysql.sock

SQL Client for Mac OS X that works with MS SQL Server

I use Eclipse's Database development plugins - like all Java based SQL editors, it works cross platform with any type 4 (ie pure Java) JDBC driver. It's ok for basic stuff (the main failing is it struggles to give transaction control -- auto-commit=true is always set it seems).

Microsoft have a decent JDBC type 4 driver: http://www.microsoft.com/downloads/details.aspx?FamilyId=6D483869-816A-44CB-9787-A866235EFC7C&displaylang=en this can be used with all Java clients / programs on Win/Mac/Lin/etc.

Those people struggling with Java/JDBC on a Mac are presumably trying to use native drivers instead of JDBC ones -- I haven't used (or practically heard of) the ODBC driver bridge in almost 10 years.

Multiple Updates in MySQL

The question is old, yet I'd like to extend the topic with another answer.

My point is, the easiest way to achieve it is just to wrap multiple queries with a transaction. The accepted answer INSERT ... ON DUPLICATE KEY UPDATE is a nice hack, but one should be aware of its drawbacks and limitations:

  • As being said, if you happen to launch the query with rows whose primary keys don't exist in the table, the query inserts new "half-baked" records. Probably it's not what you want
  • If you have a table with a not null field without default value and don't want to touch this field in the query, you'll get "Field 'fieldname' doesn't have a default value" MySQL warning even if you don't insert a single row at all. It will get you into trouble, if you decide to be strict and turn mysql warnings into runtime exceptions in your app.

I made some performance tests for three of suggested variants, including the INSERT ... ON DUPLICATE KEY UPDATE variant, a variant with "case / when / then" clause and a naive approach with transaction. You may get the python code and results here. The overall conclusion is that the variant with case statement turns out to be twice as fast as two other variants, but it's quite hard to write correct and injection-safe code for it, so I personally stick to the simplest approach: using transactions.

Edit: Findings of Dakusan prove that my performance estimations are not quite valid. Please see this answer for another, more elaborate research.

MAC addresses in JavaScript

If this is for an intranet application and all of the clients use DHCP, you can query the DHCP server for the MAC address for a given IP address.

Capturing TAB key in text box

Even if you capture the keydown/keyup event, those are the only events that the tab key fires, you still need some way to prevent the default action, moving to the next item in the tab order, from occurring.

In Firefox you can call the preventDefault() method on the event object passed to your event handler. In IE, you have to return false from the event handle. The JQuery library provides a preventDefault method on its event object that works in IE and FF.

<body>
<input type="text" id="myInput">
<script type="text/javascript">
    var myInput = document.getElementById("myInput");
    if(myInput.addEventListener ) {
        myInput.addEventListener('keydown',this.keyHandler,false);
    } else if(myInput.attachEvent ) {
        myInput.attachEvent('onkeydown',this.keyHandler); /* damn IE hack */
    }

    function keyHandler(e) {
        var TABKEY = 9;
        if(e.keyCode == TABKEY) {
            this.value += "    ";
            if(e.preventDefault) {
                e.preventDefault();
            }
            return false;
        }
    }
</script>
</body>

How to set background color of HTML element using css properties in JavaScript

KISS Answer:

document.getElementById('element').style.background = '#DD00DD';

Why can't I have abstract static methods in C#?

We actually override static methods (in delphi), it's a bit ugly, but it works just fine for our needs.

We use it so the classes can have a list of their available objects without the class instance, for example, we have a method that looks like this:

class function AvailableObjects: string; override;
begin
  Result := 'Object1, Object2';
end; 

It's ugly but necessary, this way we can instantiate just what is needed, instead of having all the classes instantianted just to search for the available objects.

This was a simple example, but the application itself is a client-server application which has all the classes available in just one server, and multiple different clients which might not need everything the server has and will never need an object instance.

So this is much easier to maintain than having one different server application for each client.

Hope the example was clear.

Big O, how do you calculate/approximate it?

Familiarity with the algorithms/data structures I use and/or quick glance analysis of iteration nesting. The difficulty is when you call a library function, possibly multiple times - you can often be unsure of whether you are calling the function unnecessarily at times or what implementation they are using. Maybe library functions should have a complexity/efficiency measure, whether that be Big O or some other metric, that is available in documentation or even IntelliSense.

Best ways to teach a beginner to program?

You could try using Alice. It's a 3D program designed for use in introductory programming classes.

The two biggest obstacles for new programmers are often:

  • syntax errors
  • motivation (writing something meaningful and fun rather than contrived)

Alice uses a drag and drop interface for constructing programs, avoiding the possibility of syntax errors. Alice lets you construct 3D worlds and have your code control (simple) 3D characters and animation, which is usually a lot more interesting than implementing linked lists.

Experienced programmers may look down at Alice as a toy and scoff at dragging and dropping lines of code, but research shows that this approach works.

Disclaimer: I worked on Alice.

Calling a function of a module by using its name (a string)

locals()["myfunction"]()

or

globals()["myfunction"]()

locals returns a dictionary with a current local symbol table. globals returns a dictionary with global symbol table.

What is Inversion of Control?

What is Inversion of Control?

If you follow these simple two steps, you have done inversion of control:

  1. Separate what-to-do part from when-to-do part.
  2. Ensure that when part knows as little as possible about what part; and vice versa.

There are several techniques possible for each of these steps based on the technology/language you are using for your implementation.

--

The inversion part of the Inversion of Control (IoC) is the confusing thing; because inversion is the relative term. The best way to understand IoC is to forget about that word!

--

Examples

  • Event Handling. Event Handlers (what-to-do part) -- Raising Events (when-to-do part)
  • Dependency Injection. Code that constructs a dependency (what-to-do part) -- instantiating and injecting that dependency for the clients when needed, which is usually taken care of by the DI tools such as Dagger (when-to-do-part).
  • Interfaces. Component client (when-to-do part) -- Component Interface implementation (what-to-do part)
  • xUnit fixture. Setup and TearDown (what-to-do part) -- xUnit frameworks calls to Setup at the beginning and TearDown at the end (when-to-do part)
  • Template method design pattern. template method when-to-do part -- primitive subclass implementation what-to-do part
  • DLL container methods in COM. DllMain, DllCanUnload, etc (what-to-do part) -- COM/OS (when-to-do part)

What's the safest way to iterate through the keys of a Perl hash?

A few miscellaneous thoughts on this topic:

  1. There is nothing unsafe about any of the hash iterators themselves. What is unsafe is modifying the keys of a hash while you're iterating over it. (It's perfectly safe to modify the values.) The only potential side-effect I can think of is that values returns aliases which means that modifying them will modify the contents of the hash. This is by design but may not be what you want in some circumstances.
  2. John's accepted answer is good with one exception: the documentation is clear that it is not safe to add keys while iterating over a hash. It may work for some data sets but will fail for others depending on the hash order.
  3. As already noted, it is safe to delete the last key returned by each. This is not true for keys as each is an iterator while keys returns a list.

What is recursion and when should I use it?

The simplest definition of recursion is "self-reference". A function that refers to itself, i. e. calls itself is recursive. The most important thing to keep in mind, is that a recursive function must have a "base case", i. e. a condition that if true causes it not to call itself, and thus terminate the recursion. Otherwise you will have infinite recursion:

recursion http://cart.kolix.de/wp-content/uploads/2009/12/infinite-recursion.jpg

How can we generate getters and setters in Visual Studio?

If you're using ReSharper, go into the ReSharper menu → CodeGenerate...

(Or hit Alt + Ins inside the surrounding class), and you'll get all the options for generating getters and/or setters you can think of :-)