How to solve “Unable to load native-hadoop library” in Eclipse

How to solve “Unable to load native-hadoop library” in Eclipse

Having just configured Eclipse for Hadoop development, I ran into the following problem:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Running jobs via the hadoop cli command worked fine; this only happened when I tried to run jobs directly from Eclipse, in local mode. After doing a little investigation, I found that the reason behind this is a java property called java.library.path that did not include the correct path.

Running from the hadoop cli command, the java.library.path property was properly set to /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native (I am using the CDH 4.2.0 distribution of Hadoop). When the job was started from inside Eclipse, the java.library.path held it’s system default value:

In order to correctly set this property you can configure Eclipse to load the Java Virtual Machine with this setting, or (and this is the better way) add the native library under the respective library from the Java Build Path. In order to do this, first right click on your project and open the Buid Path configuration screen:

In this screen, find the hadoop-common library, expand the row and add the native library by pointing to the correct location:

That’s it !

Courtesy : http://pastiaro.wordpress.com/2013/03/18/how-to-solve-unable-to-load-native-hadoop-library-in-eclipse/

Log4j, java library that specializes in logging

Log4j, java library that specializes in logging

Log4j is a Java library that specializes in logging. At the time of this writing, its home page is at http://logging.apache.org/log4j/1.2/index.html . The Java library itself is available for download at http://logging.apache.org/log4j/1.2/download.html .  At its most basic level, you can think of it as a replacement for System.out.println's in your code.

Steps to Integrate Log4j with your Java project

1. Download log4j.jar from http://logging.apache.org/log4j/1.2/download.html

2. Add the jar to your project

3. Create a new file 'log4j.properties' in your  project 'src'  and add the contents as follows and save it :

# Root logger option

log4j.rootLogger=INFO, file

# Direct log messages to a log file

log4j.appender.file=org.apache.log4j.RollingFileAppender

log4j.appender.file.File=logs/testlog.log

log4j.appender.file.MaxFileSize=1MB

log4j.appender.file.MaxBackupIndex=1

log4j.appender.file.layout=org.apache.log4j.PatternLayout

log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

4. Append one line code to integrate Log4j to the class, declare a variable say 'log' as type 'Logger'

 static Logger log = Logger.getLogger(ClassName.class.getName());

'ClassName' is the name of the class that you need to use Log4j for logging.

5. Now you can use 

 log.debug("This is debug message");

  log.info("This is info message");

  log.warn("This is warn message");

  log.fatal("This is fatal message");

  log.error("This is error message");

for debug ,info,warning,fatal and error logs instead of  System.out.println 

Thats It !

For example :

import org.apache.log4j.Logger;
public class Test {
  static Logger log = Logger.getLogger(Test.class.getName());
  public static void main(String[] args) {
   try {
   int i = 1;
   int b = 1 - 1;
   b = i / b;
  } catch (Exception e) {
   log.error("failed", e);
   e.printStackTrace();
  }
 }
}

Sent Email using Java

Sent Email using Java and gmail

import java.util.Properties;
import javax.mail.*;
import javax.mail.internet.*;

public class Mail {

public static void main(String[] args) {

// Set up the SMTP server.
String to = "mailId";// Reciver Address.
Properties props = new Properties();
props.put("mail.smtp.host", "smtp.gmail.com");
props.put("mail.smtp.auth", "true");
props.setProperty("mail.smtp.port", "587");
props.put("mail.smtp.starttls.enable", "true");
Session session = Session.getDefaultInstance(props,
new javax.mail.Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("yourEmailId", "password");// SenderID and Password.
}
});
try {
MimeMessage message = new MimeMessage(session);
message.setFrom(new InternetAddress("SenderMailId"));// Sender Id.
message.addRecipient(Message.RecipientType.TO, new InternetAddress(
to));
message.setSubject("Mail Subject");
message.setText("Mail Body");
// send message.
Transport.send(message);
System.out.println("message sent successfully");
} catch (MessagingException e) {
throw new RuntimeException(e);

}

}
}

POST request to the server using Terminal (Linux)

POST request to the server using Terminal 

curl -d "parameter1=value1&parameter2=value2" page_url

  Example:
 
  page located at http://www.formpost.com/getthis/
 
        <form action="post.cgi" method="post">
        <input name=user size=10>
        <input name=pass type=password size=10>
        <input name=id type=hidden value="blablabla">
        <input name=ding value="submit">
        </form>
 
  We want to enter user 'foobar' with password '12345'.
 
  To post to this, you enter a curl command line like:
 
curl -d "user=foobar&pass=12345&id=blablabla&ding=submit" http://www.formpost.com/getthis/post.cgi

Find and Kill a Process that is Using a Particular Port in Ubuntu

STOP TOMCAT SERVER USING TERMINAL

1- Find what application/process is using :

sudo netstat -lpn |grep :8080

and press Enter.

You will get an output similar to this one

tcp6       0      0 :::8080                 :::*                    LISTEN      6782/java

2- I have got the process Id, which is 6782, now this is the process that is using port 8080.

3- Kill the process, type:kill 6782

kill -9 6782

and now port 8080 is free.

EventSource's response has a charset ("iso-8859-1") that is not UTF-8. Aborting the connection Error

SERVER SENT EVENTS, HTML5 AND TOMCAT 7

If you test, you can understand that there is no problem while running the project with Tomcat 6.

Yes, the problem is with Tomcat 7. It supports charset : iso-8859-1 by default.

So, we need to fix this problem with describing that we are using character encoding(charset)  UTF-8.

Change the responce.setContentType as follows:

response.setContentType("text/event-stream, charset=UTF-8");

Thats it.

Where is .bashrc ?

Where is .bashrc ?

Don't forget it is a hidden file inside your home directory (you would not be the first to do a ls -l and thinking it is not there).

ls -la ~/ | more


There should be a .bashrc on the first page. If not just create it with
vi ~/.bashrc


and add in the lines you need to add into it.

Permissions of my .bash.rc are

-rw-r--r-- 1 discworld discworld 3330 Mar 10 16:03 .bashrc

(chmod 644 .bashrc to make it rw r r)