Hashtable vs HashMap vs ConcurrentHashMap

In this blog post, I am noting down the key differences between Hashtable, HashMap and ConcurrentHashMap.

	HashTable	HashMap	ConcurrentHashMap
1	It is a legacy class which as a part of Java 2 was retrofitted to implement Map interface, making it a member of java Collections framework.	It is a member of Java collections framework since Java 2.	It is a member of java collections framework  since Java 5
2	Belongs to java.util package	Belongs to java.util package	Belongs to java.util.concurrent package
3	This class extends Dictionary class, which extends Object.	This class extends AbstractMap, which extends Object.	This class extends AbstractMap, which extends Object.
4	Implemented interfaces: Serializable, Cloneable, Map	Implemented interfaces: Serializable, Cloneable, Map	Implemented interfaces: Serializable, ConcurrentMap, Map
5	Does not permit null key or value.	Permits null values and null key.	Does not permit null key of value.
6	Synchronized = threadsafe	Unsynchronized = not thread safe. External synchronization needed if multiple threads access a HashMap concurrently. Can be synchronized by Map m = Collections.synchronizedMap( new HashMap(…));	Synchronized = threadsafe. Provides better scalability than hashtable.  Does not lock entire table.
7	Elements can be iterated using enumeration as well as Iterator.	Can be iterated only with Iterator.	Can be iterated only with Iterator
8	Slow compared to HashMap if used for single threaded applications.	Fast performance compared to Hashtable for single threaded applications.
9		HashMap makes no guarantee as to the order in the map. So in the event you would want predictable iteration order (which is insertion order by default), you could use one of the subclasses of hashmap called LinkedHashMap.

More description to the points above:

5.  Hashtable and ConcurrentHashMap do not allow nulls whereas HashMap does allow null keys and values.  The main reason that nulls aren’t allowed in CocurrentMaps is if map.get(key) returns null, you cant detect whether the key explicitly maps to null vs the key isn’t mapped at all. In a non-concurrent map, you can check via map.contains(key), but in a concurrent one, the map might have changed between calls.

7. Enumeration is not fail-fast whereas Iterator is fail-fast; if the map is structurally modified at any time after the iterator is created, in any way except through the iterator’s own remove method, the iterator will throw a ConcurrentModificationException. The failing-fast behavior of iterators is not guaranteed as it is impossible to make hard guarantees in the presence of unsynchronized concurrent modification. (This is more on difference between the Enumeration and Iterator)

References: Java Docs

RabbitMQ 101

RabbitMQ was originally developed to support AMQP.  Lets see briefly what AMQP is and some introduction to its terminology:

What is AMQP?

• AMQP (Advanced Message Queuing Protocol) is a networking protocol that enables conforming client applications to communicate with conforming messaging middleware brokers.
• Messaging brokers receive messages from publishers (applications that publish them, also known as producers) and route them to consumers (applications that process them).
• Since AMQP is a network protocol, the publishers, consumers and the broker can all reside on different machines.
• The AMQP 0-9-1 Model has the following view of the world: messages are published to exchanges, which are often compared to post offices or mailboxes. Exchanges then distribute message copies to queues using rules called bindings. Then AMQP brokers either deliver messages to consumers subscribed to queues, or consumers fetch/pull messages from queues on demand.

      

• When publishing a message, publishers may specify various message attributes (message meta-data). Some of this meta-data may be used by the broker, however, the rest of it is completely opaque to the broker and is only used by applications that receive the message.
• Networks are unreliable and applications may fail to process messages therefore the AMQP model has a notion of message acknowledgements: when a message is delivered to a consumer the consumer notifies the broker, either automatically or as soon as the application developer chooses to do so. When message acknowledgements are in use, a broker will only completely remove a message from a queue when it receives a notification for that message (or group of messages).
• In certain situations, for example, when a message cannot be routed, messages may be returned to publishers, dropped, or, if the broker implements an extension, placed into a so-called "dead letter queue". Publishers choose how to handle situations like this by publishing messages using certain parameters.
• Queues, exchanges and bindings are collectively referred to as AMQP entities.

Exchange and Exchange Types:

Exchanges are AMQP entities where messages are sent. Exchanges take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings. AMQP 0-9-1 brokers provide four exchange types. Besides the exchange type, exchanges are declared with a number of attributes, the most important of which are Name, Durability (exchanges survive broker restart), Auto-delete (exchange is deleted when all queues have finished using it), Arguments (these are broker dependent). Exchanges can be durable or transient. Durable exchanges survive broker restart whereas transient exchanges do not (they have to be re-declared when the broker comes back online.)

1. Default exchange: T he default exchange is a direct exchange with no name (empty string) pre-declared by the broker. It has one special property that makes it very useful for simple applications: every queue that is created is automatically bound to it with a routing key which is the same as the queue name.

• For example, when you declare a queue with the name of "search-indexing-online", the AMQP broker will bind it to the default exchange using "search-indexing-online" as the routing key. Therefore, a message published to the default exchange with the routing key "search-indexing-online" will be routed to the queue "search-indexing-online". In other words, the default exchange makes it seem like it is possible to deliver messages directly to queues, even though that is not technically what is happening.

2. Direct Exchange: A direct exchange delivers messages to queues based on the message routing key. A direct exchange is ideal for the unicast routing of messages (although they can be used for multicast routing as well). A queue binds to the exchange with a routing key K. When a new message with routing key R arrives at the direct exchange, the exchange routes it to the queue if K = R.

3. Fanout Exchange: A fanout exchange routes messages to all of the queues that are bound to it and the routing key is ignored. If N queues are bound to a fanout exchange, when a new message is published to that exchange a copy of the message is delivered to all N queues. Fanout exchanges are ideal for the broadcast routing of messages.

4. Topic Exchange: Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange. The topic exchange type is often used to implement various publish/subscribe pattern variations. Topic exchanges are commonly used for the multicast routing of messages.

Topic exchanges have a very broad set of use cases. Whenever a problem involves multiple consumers/applications that selectively choose which type of messages they want to receive, the use of topic exchanges should be considered.

5. Headers exchange: A headers exchange is designed to for routing on multiple attributes that are more easily expressed as message headers than a routing key. Headers exchanges ignore the routing key attribute. Instead, the attributes used for routing are taken from the headers attribute. A message is considered matching if the value of the header equals the value specified upon binding.

- Headers exchanges can be looked upon as "direct exchanges on steroids". Because they route based on header values, they can be used as direct exchanges where the routing key does not have to be a string; it could be an integer or a hash (dictionary) for example.

Queues:

Queues in the AMQP model are very similar to queues in other message- and task-queueing systems: they store messages that are consumed by applications. Queues share some properties with exchanges, but also have some additional properties:

• Name
• Durable (the queue will survive a broker restart)
• Exclusive (used by only one connection and the queue will be deleted when that connection closes)
• Auto-delete (queue is deleted when last consumer unsubscribes)
• Arguments (some brokers use it to implement additional features like message TTL)

Before a queue can be used it has to be declared. Declaring a queue will cause it to be created if it does not already exist. The declaration will have no effect if the queue does already exist and its attributes are the same as those in the declaration. When the existing queue attributes are not the same as those in the declaration a channel-level exception with code 406 (PRECONDITION_FAILED) will be raised.

Queue Durability:

Durable queues are persisted to disk and thus survive broker restarts. Queues that are not durable are called transient. Not all scenarios and use cases mandate queues to be durable.

Durability of a queue does not make messages that are routed to that queue durable. If broker is taken down and then brought back up, durable queue will be re-declared during broker startup, however, only persistent messages will be recovered.

Bindings:

Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q, Q has to be bound to E. Bindings may have an optional routing key attribute used by some exchange types. The purpose of the routing key is to select certain messages published to an exchange to be routed to the bound queue. In other words, the routing key acts like a filter.

What is RabbitMQ?

• Robust messaging for applications
• Easy to use
• Runs on all major OS
• Supports a huge number of developer platforms
• Open source and commercially supported.

What can RabbitMQ do for you?

RabbitMQ is a messaging broker – an intermediary for messaging. It gives applications a common platform to send and receive messages and your messages a safe place to live until received.

Messaging enables software applications to connect and scale. Messaging is asynchronous.

Supported platforms: RabbitMQ  can potentially run on any platform that Erlang supports, from embedded systems to multi-core clusters and cloud-based servers. The following platforms are supported by RabbitMQ:

• Solaris
• BSD
• Linux
• MacOSX
• TRU64
• Windows NT/95/982000/XP/Vista/ Win 7 / Win 8
• Windows Server 2003/2008/2012
• Ubuntu and debian based Linux distributions
• Fedora and RPM based Linux distributions
• OpenSuse

Following are the features provided by RabbitMQ:

Reliability:

• RabbitMQ offers a variety of features to let u trade off performance with reliability, including persistence, delivery acknowledgements, publisher confirms and high availability.
• Reliable delivery for RabbitMQ means to ensure that messages are always delivered, even encountering failure in any part of the system.  (ie whether consumer dies, publisher dies or the rabbitMQ server crashes)
• What can fail? Failure can be anything like network failure, firewalls can interrupt idle connections, the broker and client applications can experience hardware failure or software can crash at any time.
• Connection failures: In the event of connection failure, the client will need to establish a new connection to the broker. When connections fail, the client will be informed by the connection throwing an exception (or similar language construct.)
• Acknowledgements: When a connection fails, messages may be in transit between client and server – they may be in the middle of being parsed or generated, in OS buffers or on the wire. Messages in transit will be lost – they will need to be retransmitted. Acknowledgements let the server and clients know when to do this. Acknowledgements can be used in both directions – to allow a consumer to indicate to the server that it has received/processed a message and to allow the server to indicate the same thing to the producer. RabbitMQ refers to the latter case as a “confirm”. Acknowledgements and confirms indicate that messages have been received and acted upon. An acknowledgement signals both the receipt of a message and a transfer of ownership where the receiver assumes full responsibility for it.  A consumer should not acknowledge messages until it has done whatever it needs to do with them –  recorded them into database, forwarded them on, printed them or anything else. Once it does so, the broker is free to forget about the message.
• Confirms: Like acknowledgements, the broker will ‘confirm’ messages once it has taken responsibility for those messages. For routable messages, the basic.ack is sent when a message has been accepted by all the queues. For persistent messages routed to durable queues, this means persisting to disk. For mirrored queues, this means that all mirrors have accepted the message.  
• Use of acknowledgements and confirms guarantees at-least once delivery. Without acknowledgements, message loss is possible during publish and consume operations and only at-most-once delivery is guaranteed.

Flexible Routing:

Messages are routed through exchanges before arriving at queues. RabbitMQ features several built-in exchange types for typical routing logic.

Clustering:

• Several RabbitMQ servers on a local network can be clustered together, forming a single logical broker.
• All data/state required for the operation of a RabbitMQ broker is replicated across all nodes, for reliability and scaling, with full ACID properties. An exception to this are message queues, which by default reside on the node that created them, though they are visible and reachable from all nodes. To replicate queues across nodes in a cluster, see high availability section in this documentation.
• The composition of a cluster can be altered dynamically. All RabbitMQ brokers start out as running on a single node. These nodes can be joined into clusters, and subsequently turned back into individual brokers again.
• RabbitMQ brokers tolerate the failure of individual nodes. Nodes can be started and stopped at will.
• A node can be a disk node or a RAM node. RAM nodes keep their state only in memory (with the exception of queue contents, which can reside on disk if the queue is persistent or too big to fit in memory). Disk nodes keep state in memory and on disk. As RAM nodes don't have to write to disk as much as disk nodes, they can perform better. However, note that since the queue data is always stored on disk, the performance improvements will affect only resources management (e.g. adding/removing queues, exchanges, or vhosts), but not publishing or consuming speed. Because state is replicated across all nodes in the cluster, it is sufficient (but not recommended) to have just one disk node within a cluster, to store the state of the cluster safely.
• Erlang nodes use a cookie to determine whether they are allowed to communicate with each other - for two nodes to be able to communicate they must have the same cookie. The cookie is just a string of alphanumeric characters. It can be as long or short as you like.
• Erlang will automatically create a random cookie file when the RabbitMQ server starts up. The  easiest way to proceed is to allow one node to create the file, and then copy it to all the other  nodes in the cluster.

Highly available Queues:

• Queues can be mirrored across several machines in a cluster, ensuring that even in the event of hardware failure your messages are safe.
• By default, queues within a RabbitMQ cluster are located on a single node (the node on which they were first declared). This is in contrast to exchanges and bindings, which can always be considered to be on all nodes. Queues can optionally be made mirrored across multiple nodes. Each mirrored queue consists of one master and one or more slaves, with the oldest slave being promoted to the new master if the old master disappears for any reason.
• Messages published to the queue are replicated to all slaves. Consumers are connected to the master regardless of which node they connect to, with slaves dropping messages that have been acknowledged at the master. Queue mirroring therefore enhances availability, but does not distribute load across nodes (all participating nodes each do all the work).
• This solution requires a RabbitMQ cluster, which means that it will not cope seamlessly with network partitions within the cluster and, for that reason, is not recommended for use across a WAN (though of course, clients can still connect from as near and as far as needed).

Memory and Disk Alarms:

There are two circumstances under which RabbitMQ will stop reading from client network sockets, in order to prevent crashes. They are:

• When memory use goes above the configured limit.
• When disk space drops below the configured limit.

In both circumstances the server will temporarily block connections - the server will pause reading from the sockets of connected clients, which publish messages. Connection heartbeat monitoring will be disabled too.

When running RabbitMQ in a cluster, the memory and disk alarms are cluster-wide; if one node goes over the limit then all nodes will block connections.

The intent here is to stop producers but let consumers continue unaffected.

Memory alarms:

The RabbitMQ server detects the total amount of RAM installed in the computer on startup and when rabbitmqctl set_vm_memory_high_watermark fraction is executed. By default, when the RabbitMQ server uses above 40% of the installed RAM, it raises a memory alarm and blocks all connections. Once the memory alarm has cleared (e.g. due to the server paging messages to disk or delivering them to clients) normal service resumes.

The default memory threshold is set to 40% of installed RAM. Note that this does not prevent the RabbitMQ server from using more than 40%, it is merely the point at which publishers are throttled. Erlang's garbage collector can, in the worst case, cause double the amount of memory to be used (by default, 80% of RAM). It is strongly recommended that OS swap or page files are enabled.

Disk Alarms:

RabbitMQ will block producers when free disk space drops below a certain limit. This is a good idea since even transient messages can be paged to disk at any time, and running out of disk space can cause the server to crash. By default RabbitMQ will block producers, and prevent memory-based messages from being paged to disk, when free disk space drops below 50MB. This will reduce but not eliminate the likelihood of a crash due to disk space being exhausted. In particular, if messages are being paged out rapidly it is possible to run out of disk space and crash in the time between two runs of the disk space monitor.

Who is using RabbitMQ today?

VMware: Makes extensive use of RabbitMQ in its virtualization products and cloud services. 

Google: The open-source Rocksteady project uses RabbitMQ and complex event processing to analyse user defined metrics. Its goal is to allow root cause diagnosis of breakages in real time. 


UIDAI, Government of India:  UIDAI is the largest online identity project in the world aiming to provide each of India's 1.2 billion residents with a unique identity number. UIDAI uses RabbitMQ to decouple sub-components of its application allowing it to scale.

Mozilla use RabbitMQ in Pulse, their in-house eventing and pub-sub bus.

OpenStack is an open-source initiative that provides a massively scalable cloud operating system and uses RabbitMQ for messaging.

RabbitMQ is widely used at AT&T Interactive, the local search provider.