Spring Boot – Apache Qpid Starter

I decided to Spring-ify my previous attempt at configuring and running a HelloWorld program using Apache Qpid which was set up locally and you can read all about that here: Apache Qpid setup for dummies.

In order to use Spring Boot, I modified the code (mostly followed autoconfiguration code from https://github.com/tabish121/qpid-jms-spring-boot/tree/master/qpid-jms-spring-boot-autoconfigure):

1. Use Spring Boot autoconfiguration to load JMS connection factory details
2. Schedule a simple message to be delivered to the queue every 5 seconds (@EnableScheduling is wonderful)
3. A JMS Listener to receive the messages and log to the console.

 

The modified code can be found at https://github.com/trishulpani/qpid-helloworld/tree/helloqpidspringboot

 

Advertisements

QPID message broker local set up for dummies like me

Apache Qpid (https://qpid.apache.org/index.html) is a messaging tool that supports AMQP amongst other things. I recently set this up locally (running on Mac OS High Sierra 10.13.3) and was overwhelmed with the documentation (it’s pretty good but no single tutorial to show the basics ) – hence this post.

I just wanted to set up a simple messaging (hello world type) infrastructure that would have the following components:

1. A Messaging Server 

This is the ‘thing’ that runs like a server and is responsible for receiving and routing messages among another mind-boggling number of things it can do. The component that does this (for Java folks) is downloadable at:

https://qpid.apache.org/components/broker-j/index.html

Click on the ‘Qpid Broker-J 7.0.1’ (latest at the time of writing), and download the tar file to a suitable location. Untar it and simply run:

<qpid server location>/bin/qpid-server

By default, the Broker starts up on HTTP port 8080 and AMQP port: 5672.

Why do these ports matter?

HTTP Port is used by the QPID Broker management console.

AMQP Port: This is required by the JMS client (the piece of code which will send a message) when creating a connection (more on this later).

However, with a little foresight, you might not want this when running locally (8080 is a commonly used port for local Tomcat and other app servers). So, you can restart the Broker by passing a different port number like so:

<qpid server location>/bin/qpid-server -prop “qpid.amqp_port=10000” -prop “qpid.http_port=10001”

You’ll know if the server started up fine if you see something similar in the console/terminal:

Now, I can access the Broker management console UI like so:

The default credential is: admin/admin

Once logged in, you should see a similar screen:

So far, so good. Now I need the other piece of the puzzle. A ‘client’ that can publish some message and consume the message.

2. Messaging client

Here’s what I followed for setting up the client:

https://github.com/apache/qpid-jms/tree/0.29.0/qpid-jms-examples

However, I didn’t download the sample as it is. I was using IntelliJ, so I created a blank Maven project. My pom.xml looks like:

Also, I changed the jndi.properties to accommodate the new port that I previously assigned:

Ok, so I changed the queue name too (I mean, who wants a queue named queue, right?)

Now, I was being dumb and ran the program and was promptly greeted with the following exception:

Caused by: javax.jms.JMSSecurityRuntimeException: No supported mechanism, or none usable with the available credentials. Server offered: [CRAM-MD5, SCRAM-SHA-1, SCRAM-SHA-256]Caused by: javax.jms.JMSSecurityRuntimeException: No supported mechanism, or none usable with the available credentials. Server offered: [CRAM-MD5, SCRAM-SHA-1, SCRAM-SHA-256] at org.apache.qpid.jms.sasl.SaslMechanismFinder.findMatchingMechanism(SaslMechanismFinder.java:103)

Oops!

The first step (like most things in life) was to ask Google if this meant anything. Nothing substantial came up. Some posts indicated that this might have to do with the SASL supported etc. A lot of BS that hardly helped.

So, I went back to the sample and actually READ through the instructions. I missed two critical steps:

a. You need to create the damn queue (remember the testqueue from jndi.properties?). Ok. How to create a queue? Google again. The first link that came up:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_MRG/2/html/Messaging_Installation_and_Configuration_Guide/Create_and_Configure_Queues_using_qpid_config.html

Nope. Not what I wanted.

Back to Qpid documentation. Found this:

https://qpid.apache.org/releases/qpid-broker-j-7.0.0/book/Java-Broker-Management-Channel-Web-Console.html

Duh! Create a queue named ‘testqueue’. Just leave the remaining defaults. I need it up and running for the time being.

Another dummy tip: Double click on the ‘defaults’ node under ‘virtualhosts’ which will bring up the following screen:

Scroll down, and you should see a ‘Add queue’ button. Click it. click it NOW!

Enter the name and you are done.

The queue was created, and you should now see another node under ‘default’ node. (Yes, you can double-click on the queue name and it will show you the details now)

b. You need to pass the following as JVM parameters in order to get rid of the error, like so:

-DUSER=guest -DPASSWORD=guest

 

 

 

Why do we need this? Well, by default the server is using ‘passwordFile’ as the authentication mechanism. So, I need to pass some credentials that the server will recognize.

How does one know the list of credentials that can be identified by the server when using this kind of authentication?

The list can be found in the  <qpid server untar’ed location>/7.0.1/etc/passwd file.

AHA! Now the exception makes so much sense !!

I got sidetracked a bit. My apologies.

But, finally, now when I ran the HelloWorld client, I got this in the console:

How did I know the client was able to send the message and receive it? Check the Broker console:

Note the ‘Delivered’ statistics.

Yup. Got this simple hello world to work and learnt a lot of terminologies in the process.

Good stuff!

Github source: https://github.com/trishulpani/qpid-helloworld

 

University Transcript from Calcutta University – Feb 2016

One might require University transcripts for a number of reasons. Transcripts are required for visa process (Canada/ Australia etc.), applying to US universities etc. The usual process is to contact the University and they should be able to help with the exact steps.

The process below is for CU (Calcutta University) alumni as of Feb 2016. This also assumes that you are available and have enough time to visit the Transcript department in-person on 2 separate days.

For people who do not/cannot visit in-person, you’ll be required to authorize someone to submit the application and collect the Transcripts on your behalf.

Prerequisites:

1. Make sure you have at least 30 days in hand.
2. Go through the following document: CU Transcript Instructions [Don’t waste your time by calling the listed phone numbers (numbers are either dysfunctional or there’s no one to pick up) or emailing ( emails are non-deliverable to both the addresses ).
3. Download the form, print and fill it up (preferably with blue ink to make your entries more visible).
4. Photocopies of Registration Certificate, All Marksheets (all semesters/years).
5. Registration Certificate and All Marksheets (Original).
6. Cash ( INR 200 per transcript ).
7. A pen.
8. Willingness to be present in-person to apply on one day (1 – 1.30 hrs depending on how busy the department is on that particular day) AND on a second day to collect the transcripts. (Sorry, they don’t have an online system even in 2016)

Location:

Transcript department is located on the 4th Floor of Darbhanga Building (inside Calcutta University Main Campus ). The nearest Metro station is Central – a 5-minute walk.

Hours:

The Transcript department is open from 11 am to 1.30 pm on weekdays.

Special Note:

1. Even though the form asks for a Demand Draft (DD) to be submitted along with the application, it’s NOT required.

Yeah! I learned it the hard way. I wasted about an hour at a bank getting the DD prepared and when I arrived at the CU Transcript department, they said CASH is preferred.

2. You need not fill up the section that requires University names and locations to which the transcripts are to be sent (assuming you will be collecting it yourself).

Application Process

1. Ensure that you are carrying the items mentioned in the Prerequisites section.
2. Head straight for the CASH section (ask anyone for directions) once you enter the CU Main Campus on College Street. Get a CHALLAN for INR 1/- (One rupee).
3. Darbhanga building is bang opposite the building that has the CASH section/department. Proceed to the 4th Floor (there are elevators available – although they are located in a very nondescript area of the ground floor. Ask someone if you are not able to find them).
4. Follow the directions to the ‘Transcript section’ – as of today, it has been relocated to the back side of the 4th Floor.
5. Once you reach the Transcript section, you’ll be asked (usually) for the purpose of your visit. If not, state politely the purpose of your visit.
6. You’ll then be asked to submit the documents. You may be asked for the originals (I wasn’t – but it’s better to carry them in any case).
7. You’ll be asked to fill up the CHALLAN (In case you forgot step 2 / didn’t bring the CHALLAN, you will usually be provided one and asked to provide a blank one once you have submitted the cash at the CASH section).
8. Next, if the documents are sufficient, the person will sign and state “Please accept” on the CHALLAN which you’ll be required to take to the CASH section and pay the required amount.
9. Visit the CASH section and pay the amount. (Remember to take a Blank CHALLAN if you missed step 2 ). If you are paying in denominations of INR 500 or more, be prepared to enter denomination, note number, your phone number in a separate register which will be provided at the CASH counter.
10. The Cashier will hand you 2 pieces of paper with the particulars of payment.
11. Head back to the Transcript department and hand back one part of the particulars and Blank CHALLAN (if you missed step 2). Keep the other part for your reference which will be required on the day of collecting the transcripts.
12. You will be asked to return at a later date (usually 3 weeks from date of submission) to collect the Transcripts.

That’s it – you’re done! Pretty hassle free if you ask me.

[Important Update: 29th January 2018]

I’m glad this post helped quite a few people and thanks to the efforts of Ananya (she has her own blog here), we now have the steps to get transcripts and send to WES as follows:

Step 1: Things to take: a. A printout of the WES Academic Records Request form available here, fill part A of this form.

b. Copy of your PDC (The number of copies required is equal to the number of transcripts you have requested for).

c. Copy of your part 1, part 2 and part 3 mark sheets, whichever is applicable to you (The number of copies required is equal to the number of transcripts you have requested for).

Head straight for the RRG department on the 4th floor of the Darbhanga building at Calcutta University. Someone will fill up Part B of the WES Academic Records Request form from the RRG department for you.

Step 2: Go to the Registrar on the first floor of the same building with the form and the transcript. You will not meet the Registrar, there are people outside the office who will take the form inside for you and get the Registrar’s signature. Make sure you ask for the seal as well, somebody else outside the office will put the seal on the form once it is signed. If you don’t ask, they might not.

Step 3: Go back to the RRG department:

a. They will sign and stamp the transcript, copies of your PDC and the mark sheets.

b. They will put all this along with the WES Academic Records Request form in a University envelope and seal the envelope.

c. They will put a stamp on the back flap of the envelope.

d. The Deputy Assistant will sign on the stamp.

Step 4: Go back to the Registrar’s office on the first floor, someone will take the envelope inside and get a final signature from the Registrar and a stamp with the date.

Fairly simple, eh? You are sorted, use a good courier service to send the envelope to WES, Canada.

Laravel Homestead installation on Mac

Larval Homestead(0.33)  installation on Mac El Capitan.

Assuming that the latest Virtual Box is already installed, the following are the basic steps required to get the Laravel Homestead box up and running without any errors. Might save a bit of headache for others or for myself in the future.

1. The command as given in Laravel Official Installation docs

vagrant box add laravel/homestead

did not work for me. I downloaded the box from https://atlas.hashicorp.com/laravel/boxes/homestead/versions/0.3.3/providers/virtualbox.box

2. Run:

vagrant box add laravel/homestead <virtualbox zip file>

3. Make an entry in /etc/hosts file as

192.168.10.10  homestead.app

4. Do a vagrant up.

5. If step 3 fails with SSH timeout error, Create new ssh key-pair in a new    location : ssh-keygen -t rsa -C “you@homestead”

6. Set the SSH file locations in ~/.homestead/Homestead.yml

7. Do vagrant up – this time it should succeed.

8. Do a vagrant ssh (This should login the user to the vagrant VM)

9. Create folder structure to reflect the ‘sites’ to the section of the ~/.homestead/Homestead.yml file, typically /home/vagrant/Code/Laravel/public ( Initially, the Laravel/public folder is missing)

10. Ensure the ‘folders’ map section of the ~/.homestead/Homestead.yml file points to a legitimate directory. This will be synched with the vagrant box folder. This is the folder in the host OS( in this case Mac OS El Capitan).

11. Put a info.php with the content <?php phpinfo(); ?> inside the folder created in step 8.

Now, opening http://homestead.app/info.php should open up the php information page indicating that the setup is now complete.

Ahoy Sailor on your Laravel journey!

Installing and using Laravel 4.2 on CentOS 6.5

There have been rave reviews about Laravel and how its supposedly cooler than CodeIgniter etc. I’m new to the PHP world – and I got severely stuck while getting to the ‘You have arrived Laravel homepage’. I spent a couple of days figuring out a ‘sane’ process – and here are the overall steps:

Environment: CentOS 6.5 running on VirtualBox 4.3 (running on Mac OS X Yosemite). [I didn’t want to spend too much time fighting OS X and setting up MAMP etc.).

Firstly, stop and remove Apache HTTPD if present: [ You might need to be root in order to execute this command]

$  service httpd stop

$ yum remove httpd

Re-install HTTPD:

$ yum install httpd

For Laravel 4.2, PHP > 5.4 is required. So, remove any older PHP installations by using:

$ yum remove php-*

[This removes php and its dependencies]

Install PHP 5.6 and Laravel’s hard dependency PHP-MCrypt, PHP-MySQL (if using MySQL):

$ rpm -Uvh https://mirror.webtatic.com/yum/el6/latest.rpm
$ yum install php56w php56w-opcache php56w-mcrypt php56w-mysql

Check PHP version by using : php -v (It should now report version > = 5.6).

Next, download the Laravel master repo, from : https://github.com/laravel/laravel/archive/master.zip, Unzip  and copy extracted folder  to /var/www/<appName> folder.

Add VirtualHost entry to Apache http config file at : /etc/http.d/conf/httpd.conf

<VirtualHost *:80>
     DocumentRoot /var/www/<appName>/public
<Directory /var/www/<appName>/public > 
    AllowOverride all
  </Directory>
</VirtualHost>

Find out the Owner and Group of your Apache installation by using:

# egrep -i '^user|^group' /etc/httpd/conf/httpd.conf 
User apache
Group apache

Change the permission of  app/storage folder and allow the above user to write to it (This avoids the cryptic Exception in error handler and blank page issue after installing Laravel and trying to access http://localhost)

chown -R apache:apache app/storage

You might also need to do:

echo "0" > /selinux/enforce

Now, run the following in the folder /var/www/<appName> folder:

composer install

If PHP version is correct and everything else goes well, the folder should now contain a ‘vendor’ folder that contains all the required dependencies as described in the file composer.json.

Now, restart HTTPD server:

service httpd restart

Point your browser to localhost:80/ and this should bring up the ‘You have arrived ‘ page indicating that Laravel framework is setup properly.

Feel free to ask if you have any issues – I’ll try and help.

Cheers !!

Setting up Wordpress locally on Mac OS X

This is specifically for installing WordPress 4.0 and serving it locally using MacOS X built-in Apache.

My recent fascination with WordPress started with me heading to www.wordpress.org and downloading the zipped version. Once downloaded you have to unzip it using tar -xvf command (if unzipping from command line) – or just double click the file in Finder.

To setup WordPress, basically the following are required:

PHP ~ 5.4

MySQL

Apache HTTP server (NOT Tomcat – which is a Webserver)

These should already be available and can be easily checked.

Ensure PHP module is enabled

Go to /etc/apache2 and do a ‘vi httpd.conf’

This folder maybe write protected – so use a sudo if the it complains that the file is in read-only mode.

Uncomment the line:

#LoadModule php5_module libexec/apache2/libphp5.so

and ‘press Esc -> :wq’

That’s it. Now restart apache using ‘sudo apachectl restart’ and you’re good to go.

Just to check if PHP was enabled properly, create a file called info.php at /Library/WebServer/Documents folder and put the following in the file:

<?php phpinfo(); ?>

If you open a browser, and hit the URL : http://localhost/info.php, it should now spit out the entire PHP environment details thereby confirming that the PHP enablement in Apache had gone fine.

Prepare MacOS Apache to server the wordpress content.

The default location of the files being served is /Library/WebServer/Documents. If you can’t find the folder in Finder, Press Cmd + Shift + G, and then enter the location manually.

Next, either copy the unzipped WordPress folder in this directory OR make a new folder inside the above folder (say xyz) and paste the unzipped WordPress folder contents into this.

Now, head over to http://codex.wordpress.org/Installing_WordPress and follow the steps. Just make sure the DB connection details are correct.

WordPress is installed successfully. Enjoy !

Standalone client to send JMS messages to a JBOSS queue via JNDI

Pretty trivial exercise. But needs a few steps and unless you know exactly what you are doing, it might cause a lot of pain.

Firstly, you’d need the JMS connection factory and queues configured in the local JBoss. This is pretty easy to do.

A client jar is required for the InitialContext located at <JBOSS-HOME>\bin\client\jboss-client.jar

Secondly, you’d need something of the following:

import java.io.File;
import java.io.IOException;
import java.util.Collection;
import java.util.Hashtable;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.InitialContext;
import javax.naming.NamingException;

import org.apache.commons.io.FileUtils;
import org.apache.commons.io.filefilter.DirectoryFileFilter;
import org.apache.commons.io.filefilter.WildcardFileFilter;

public class LocalJBOssMQSender {

    /**
     * @param args
     */
    public static void main(String[] args) {

        try {

            Hashtable< String, String > envProps = new Hashtable();
            envProps.put("java.naming.factory.initial", "org.jboss.naming.remote.client.InitialContextFactory");
            envProps.put("java.naming.provider.url", "remote://127.0.0.1:4447");
            envProps.put("java.naming.security.principal", "xxxx");
            envProps.put("java.naming.security.credentials", "xxxxxx");

            InitialContext ic = new InitialContext( envProps );

            ConnectionFactory cf = (ConnectionFactory)ic.lookup("jms/<connection factory name>");
            Queue inboundQueue= (Queue)ic.lookup("jms/queue/<QueueName>");

            Connection connection = cf.createConnection();

            Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            MessageProducer producer = session.createProducer(inboundQueue);

            //pick up all .xml files in a given directory and send them as messages to the queue.
             Collection<File> files = FileUtils.listFiles(new File("C:\\temp\\test"), 
                        new WildcardFileFilter("*.xml"), DirectoryFileFilter.DIRECTORY);

                for(File file : files){

                    System.out.println(file.getName());

                    String messageStr = FileUtils.readFileToString(file);
                    TextMessage message = session.createTextMessage(messageStr);

                    //message.setStringProperty("messageType", "Presentation");
                    // Start the connection
                    connection.start();

                    producer.send(message);
                    System.out.println("Sent message:\\n" + message);
            }

                producer.close();
                session.close();
                connection.close();

        } catch (NamingException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (JMSException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }

}

Entrust issue on IBM WAS 8.0.5

My most recent challenge:

A perfectly working piece of code running on JBoss 7 that uses encryption using Entrust , fails with the following on IBM WAS 8.0.5 –

Caused by: com.entrust.toolkit.exceptions.PKCS7Exception: internal error at com.entrust.toolkit.b.run(Unknown Source)at java.lang.Thread.run(Thread.java:772) Caused by: java.lang.RuntimeException: Error from EncodeListener: Unable to calculate encrypted digest: RSA signature failed to initialize for signing: Caught an attempt to access key material in a confined key. at iaik.asn1.DerCoder.encode(Unknown Source) at iaik.asn1.DerCoder.encodeTo(Unknown Source) at iaik.asn1.DerCoder.encodeTo(Unknown Source) at iaik.pkcs.pkcs7.ContentInfoStream.writeTo(Unknown Source)

So, I started out by trying to go through the usual route. Googling didn’t help – nor could I understand what the exception was trying to say.

We built a standalone servlet , deployed on a local WAS trial version and it worked. The local WAS was version 8.5. Deployed the servlet on WAS 8.0.5 – same error.  The only difference was that the cert files were bundled in a JAR file instead of being in the app’s WEB-INF\classes folder.

Changed that in the app, and it began to work !

Or so i thought. The actual issue was finally found to be a completely different. It had to do with how our apps were packaged. In the actual environment we had 2 web apps having the same common utility jar being bundled individually with the web apps. However, this common piece of encryption util was @Autowired – Spring’s way of telling that the component will be ready to use as soon as the apps were deployed. Now, this auto-wiring was happening for the other web app and once loaded by the classloader, this wasn’t available for loading again when it was actually required by the other app. Hence the issue. We changed the bean definition to load laziy in the context file with lazy-load=”true” and that finally solved the issue.

Sidenote: we raised this issue with IBM’s EMR – and the response was that we were using a third party library not supported by IBM ! Take that IBM – you don’t even know what works in your app server and what doesn’t !!

Centos 6.4 in VirtualBox VM on Mac OS Lion + Windows + SSH

Mac OS Lion handles VirtualBox VMs much better than Windows does on a dual core AMD laptop. Everything runs faster and there’s no visible lag in running stuff inside the VM and the host. Its amazing !

Anyway, I decided to fiddle around with installing CentOS 6.4 as a guest OS on Oracle VirtualBox. The installation is a breeze. Here’s a good guide: http://opensourcedbms.com/dbms/installing-cent-os-6-3-linux-in-oracle-virtual-box-ssh-access/.

I skipped the port forwarding thing as its not required.

Next up, was enabling SSH between Mac OS and the guest OS. Here’s what saved me a lot of trouble: http://wiredrevolution.com/virtualbox/setup-ssh-access-between-virtualbox-host-and-guest-vms

However, the only thing that gave me issues was that I wasn’t able to add a ‘Host Only adapter’ from the settings of the individual VM. Firstly it complained that ‘Incorrect configuration detected’. Puzzling. Then I read somewhere that I needed to ‘add’ a Host only network adapter from VirtualBox -> Preferences ->Network and click the green button that adds ‘vboxnet0’ as an adapter. After this I followed the above guide and I was able to SSH into the machine from Mac OS.

Could I push it further and access this CentOS guest from another Windows guest running on VirtualBox? (Yes, I could finally run multiple guest OSes and yet nothing slowed down). I installed Putty on my Windows guest and there it was – as easy as pie to access CentOS via SSH. Cool.

This meant I could share files via SCP between my guest and host VMs. Awesome !

Installing Chrome on Fedora 18

Installed Fedora 18 on a VirtualBox VM on my Mac and it works great. I installed the KDE spin. Although am liking the interface, I don’t like Konquerer much. Chrome is insanely fast even inside a VM. So, I decided to install Chrome.

Note: I installed a Fedora 64 bit, so the steps here are for 64 bit Chrome

It’s really easy.

1. Add Google YUM repository details (used for checksum etc.)

Create or edit the file /etc/yum.repos.d/google-chrome.repo and add the following lines:

[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/x86_64
enabled=1
gpgcheck=1

2. Execute the following command as root

yum install google-chrome-stable

You should now have Chrome as an option in Applications -> Internet.

But, here comes a surprise – Chrome cannot be run as root by default. So it won’t work unless you do the following.

1- open Konsole Terminal
2- type

cd /opt/google/chrome/
hexedit chrome

(if you dont have hexedit or  it says hexedit command not found, just do a

yum install hexedit

)

3- you will see hex editor program press Tab to switch to ascii
4-press ctrl+s to search for “geteuid” and modify it to “getppid”
5- ctrl+X to exit then press y for saving changes

Original solution at: http://www.backtrack-linux.org/forums/showthread.php?t=43560

Thats it ! You should now be able to run your favorite Chrome as root 🙂