Google

Apr 23, 2013

Understanding the MapReduce concept

Q. What is a MapReduce?
A. On a high level MapReduce involves 2 steps

  • Step 1 is mapping: Take one input, process into many outputs. For example, take some wooden blocks and split them into smaller blocks. Process them to have key/value pairs.
  • Step 2 is reducing: Process those many outputs, for example process each block by coloring them in different colors and then generate one output by combining the split blocks with similar color into one block by gluing them together.



Why do we need to map it and then reduce it? Because if the data sets are so huge, you can process them in parallel by different machines by splitting up the tasks to make it more efficient.  The data is split across server nodes. This is the MapReduce way of processing large data. Not every piece of data can be easily split and then combined.

Example:  Input = "big brown fox jumped over a brown fence". The requirement is to count the number of occurrences of the word "count".

Mapping phase:

1. Break the input "big brown fox jumped over a brown fence" into individual word and map them into key value pairs [key, value]. For example [big, 1] [brown, 1], [fox, 1] , [jumped, 1], [over, 1], [a,1], [brown, 1], [fence, 1].

2. Sort them by the key.

3. Send all the mapped ‘brown’ key/value pairs to the same reducer node, joining on the values e.g. ‘brown’ => [1,1] where for each occurrence of ‘brown’, we are aggregating it’s count of 1.


Reducing phase:

Count up all the values for the each word, so that e.g. ‘brown’ appears 2 times


"Facebook created Cassandra, Google created BigTable and MapReduce, Amazon created SimpleTable and LinkedIn created Project Voldemort", etc. We know there are quite a few solutions out there.

Labels: ,

Apr 19, 2013

Handy DOS commands for Java developer


Q. How will you copy files more robustly in DOS?
A. Using the Robocopy (stands for Robust File Copy) or Xcopy command line utilities. Check for its options with "robocopy /?".

Q. How will you remotely connect to a remote Windows or NT server?
A. You can use the "mstsc" command from the run window. This will allow you to connect to a remote box.



Q. In DOS, how to recursively delete subversion or (.SVN) folders?
A. In DOS (i.e. Windows), you can recursively delete the .svn folder with the following command

for /d /r . %d in (.svn) do @if exist "%d" rd /s/q "%d"

For example:

C:\projects\myapp-parent\trunk>for /d /r . %d in (.svn) do @if exist "%d" rd /s/q "%d"


Note: In Windows under  file explore, the hidden files might not be displayed by default. ".svn" is a hidden file, and to turn it on you need to click on Tools --> Folder Options and then select "View" and ensure that "Show hidden files and folders" is selected.
Q. How will you list  the ip address of your machine?
A.Use the ipconfig command


C:\ipconfig


Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : dhcp.mydomain.suffix

   IPv4 Address. . . . . . . . . . . : 10.102.191.2

   Subnet Mask . . . . . . . . . . . : 255.255.255.0

   Default Gateway . . . . . . . . . : 10.102.199.1


Q. How will you list  your computer name?
A. use the hostname command.

Q. How will you list the environment properties?
A. You type the set command. This will display all the environment variables.

If you want to list the PATH variable then type

c:\set path


For variables starting with user or lib, you can type

c:\set user 

c:\set lib


and so on. if you want to send the output to a file

c:\set lib > file.txt

and then you can view the file

c:\type file.txt

c:\notepad file.txt


If you want a particular environment variable like "path" then try

c:\echo %path%

c:\echo %JAVA_HOME%


Q. How will you get the name of your computer?
A. With the hostname command.

Q. How will you check if a remote host is reachable?
A. ping

Q. How will you check for the network statistics like what ports are established, etc?
A. With netstat command. To get all possible optional parameters get help with "netstat /?"


Q. Can you list some handy commands useful for developers?
A.

1. assoc command to find out  to what application a particular file extension is mapped to.

c:\>assoc .zip
.zip=WinZip

c:\>assoc .doc
.doc=Word.Document.8

2.tree command  to view your maven directory or project directory structure. You could pipe this command to a txt file.

c:\projects\my-app>tree > x.txt
c:\projects\my-app>type x.txt


3. fc to compare two files.

c:\projects\aes-gui>fc file.txt file2.txt

4. tasklist to list all the tasks with PID, Mem Usage, etc.

5.  The net command can be used to map a network drive.

c:\projects\aes-gui>net use z: \\remotecomputer\folder
To display all the network drives try
c:\projects\aes-gui>net use 

Labels:

Apr 17, 2013

RESTFul Web Service URI conventions with Spring MVC examples

The high level pattern for the RESTful URI is
  • http(s)://myserver.com:8080/app-name/{version-no}/{domain}/{rest-reource-convetion}
For example:

  • http(s)://myserver.com:8080/accounting-services/1.0/forecasting/accounts
to list  all the accounts. This is a plural resource returning a collections of accounts. The URI contains nouns representing the resources in a hierarchical structure. For example, if you want a to get a particular transaction value under an account you can do

  • http(s)://myserver.com:8080/accounting-services/1.0/forecasting/account/123/transaction/567
Where  123 is the account number and 567 is the transaction number or id. This is a singular resource returning a single transaction.

What if you want to list a collection of transactions that are greater than a particular date?

  • http(s)://myserver.com:8080/accounting-services/1.0/forecasting/account/123/transactions/search?txn-date=20120201
The  verbs are defined via the HTTP methods GET, POST, PUT, and DELETE. The above examples are basically GET requests returning accounts or transactions. If you want to create a new transaction request, you do a POST with the following URL.

  • http(s)://myserver.com:8080/accounting-services/1.0/forecasting/account/123/transaction
 The actual transaction data will be sent in the body of the request as JSON data. The above URI will be used with a PUT http method to modify an existing transaction record.



Finally, you can also control which method gets executed with the help of HTTP headers or host names in the URL. Let's see some Spring MVC examples in a controller class as to how it maps a request URI, headers, etc to execute the relevant method on the server side.


Here is the sample code with GET requests




@Controller
@RequestMapping("/forecasting")
public class CashForecastController

{

    @RequestMapping(
            value = "/accounts,
            method = RequestMethod.GET,
            produces = "application/json")
    @ResponseBody
    public AccountResult getAllAccounts(HttpServletResponse response) throws Exception
    {    
        //get the accounts via a service and a dao layers
    }  

 @RequestMapping(
            value = "/accounts.csv,
            method = RequestMethod.GET,
            produces = "text/csv")
    @ResponseBody
    public void getAllAccounts(HttpServletResponse response) throws Exception
    {    
        //produces a CSV download file
    }  
 

    @RequestMapping(
            value = "/account/{accountCd}",
            method = RequestMethod.GET,
            produces = "application/json")
    @ResponseBody
    public Account getAccount(
            @PathVariable(value = "accountCd") String accountCode, HttpServletResponse response) throws Exception
    {
        //get the accounts via a service and a dao layers
 }
 
 
 //accept only if there is a special header
 @RequestMapping(
            value = "/account/{accountCd}",
            method = RequestMethod.GET,
    headers =
            {
                "operation=special"
            }
            produces = "application/json")
    @ResponseBody
    public Account getAccountSpecial(
            @PathVariable(value = "accountCd") String accountCode, HttpServletResponse response) throws Exception
    {
        //get the accounts via a service and a dao layers
  //special handling based
 }

    @RequestMapping(
            value = "/account/{accountCd}/transaction/{transactionId}",
            method = RequestMethod.GET,
            produces = "application/json")
    @ResponseBody
    public Transaction getTransaction(
            @PathVariable(value = "accountCd") String accountCode, 
   @PathVariable(value = "transactionId") String txnId, 
   HttpServletResponse response) throws Exception
    {
        //get the accounts via a service and a dao layers
  //accountCode and txnId can be used here
 }
 
 @RequestMapping(
            value = "/account/{accountCd}/transactions/search",
            method = RequestMethod.GET,
            produces = "application/json")
    @ResponseBody
    public TransactionResult getTransactions(
            @PathVariable(value = "accountCd") String accountCode, 
   @RequestParam(value = "txn-date", required = true) @DateTimeFormat(pattern = "yyyyMMdd") Date txnDate,
   HttpServletResponse response) throws Exception
    {
        //get the accounts via a service and a dao layers
  //accountCode and txnDate can be used here
 }
} 


Here is the sample code with POST and PUT requests
@Controller
@RequestMapping("/forecasting")
public class CashForecastController

{
    @RequestMapping(
            value = "/account/transaction",
            method = RequestMethod.POST)
    public @ResponseBody Transaction addTransaction(@RequestBody Transaction txn, HttpServletResponse response)
            throws Exception
    {
       
        //logic to create a new Transaction records via service and dao layers
        
    }
 
 
  @RequestMapping(
            value = "/account/transaction",
            method = RequestMethod.PUT)
    public @ResponseBody Transaction modifyTransaction(@RequestBody Transaction txn, HttpServletResponse response)
            throws Exception
    {
       
        //logic to modify a Transaction record via service and dao layers
        
    }
} 


Do's and Don'ts

  • Don't use query parameters to alter state. Use query parameters for sub-selection of a resource like pagination, filtering, search queries, etc
  • Don't use implementation-specific extensions in your URIs (.do, .py, .jsf, etc.). You can use .csv, .json, etc.
  • Don't ever use GET to alter state. Use GET for as much as possible. Favor POST over PUT when in doubt. 
  • Don't perform an operation that is not idempotent with PUT. 
  • Do use DELETE in preference to POST to remove resources.
  • Don't clutter your URL with verbs or stuff that should be in a header or body. Move stuff out of the URI that should be in an HTTP header or a body. Whenever it looks like you need a new verb in the URL, think about turning that verb into a noun instead. For example, turn 'activate' into 'activation', and 'validate' into 'validation'.

Labels: , ,

Apr 16, 2013

RESTClient tool to test RESTful web services

Related posts to test web services

The http://www.wiztools.org/ has some handy open source Java tools like RESTClient, Regular Expression Tester, etc. RESTClient tool is GUI based and a good alternative to the Unix command line based tool CURL and the Firefox poster plugin. SoapUI is another GUI based client for both RESTful and SOAP based web services. This blog posts shows how easy it is to get started with RESTClient tool from WizTools. The example below shows a POST request as the GET requests are easier to test. The diagrams below illustrates posting of JSON data. The RESTful web service operations GET, POST, PUT, and DELETE correlates with the database CRUD operations Read, Create, Update, and Delete respectively.

Step 1: From WizTools click on the link to RESTClient.




Step 2: Download the JAR file "restclient-cli-3.1-jar-with-dependencies.jar". Create a short cut to the jar file. Double clicking on the short cut will open the RESTClient GUI as shown below. You can type in the URL and select "POST" as the HTTP method.


Take note of the HTTP headers passed with the request.


Step 3: You need to select the "Content-Type" that you want to post. Select  "application/json" as the content-type.




The GET requests are very straight forward requiring only the URL.


The SoapUI tool requires a WADL file for the RESTFul web services. Here is an example of SOAP Web Service using the SoapUI tool.


Related posts to test web services

Labels: , , ,

Apr 11, 2013

Unix commands for software developers

Unix for software developers

1.Top 17 Unix commands Java developers use frequently 2.History commands 3.Shell scripting 4. Reading from a file
5.Purging older files 6.Splitting and archiving files 7.Emulator, SSH client 8.Unix commands for developers 9.Unix basic interview Q&A

Unix is very powerful, and you can achieve a lot with less amount of code. Most production systems run in a Unix environment, and it is very useful to know your commands to solve technical issues. Some handy Unix commands looping, recursive search, control characters,  and crontab.

Q. How will you go about concatenating the contents in a number of text/sql files to a single file? The account_sqls.patch file contains the following entries to a number of SQL files to be executed sequentially.

./tables/account_summary.sql
./tables/account_detail.sql
./tables/account_history.sql
./tables/account_link.sql

How will you combine the SQL contents in the above 4 files into one?
 

A. Firstly, cd to the folder where the account_sqls.patch file is, and then type the following command on a shell command line.

for sql in $(sed -n '1,4p' account_sqls.patch); do cat $sql >> accounts_combined.sql; done



1, 4p                                                                    // print lines 1 to 4 from account_sqls.patch
for sql in $(sed -n '1,4p' account_sqls.patch);  // for each line read
do cat $sql >> bulk_data_changes.sql; // append the contents of each file to a different file accounts_combined.sql.

Note: This is handy for combining contents from a number of different files.

Q. How will you recursively search for presence of a particular text under multiple folders unix? For example, search for text "CashEnum" within a java project?

A. Use the find and grep commands. The find will look for the files and the grep will filter out the files that has the text "CashEnum".

find myproject -type f -exec grep "CashEnum" {} /dev/null \;

  • -type f --> find files. 
  • the addition of /dev/null is so that grep will consistently print the file name when it finds a match.

The above command will search all file extensions, including .jar. You could restrict it to look for only java files as shown below

find myproject -type f -name *.java -exec grep "CashEnum" {} /dev/null \;


-name *.java --> looks for only java files.

Q.How will you go about executing multiple maven commands on unix? For example, build two separate projects one after another, for example -- project1 and project2. The project2 to should only build if project1 successfully builds.

A. Use the && control operator to combine two commands so that the second is run only if the first command returns a zero exit status. In other words, if the first command runs successfully, the second command runs. If the first command fails, the second command does not run at all. For example:

$ mvn --file project1/pom.xml clean install -Dmaven.test.skip && mvn --file project2/pom.xml clean install -Dmaven.test.skip

Similarly, the || control operator separates two commands and runs the second command only if the first command returns a non-zero exit status, that is if it fails. In other words, if the first command is successful, the second command does not run. This operator is often used when testing for whether a given directory exists and, if not, it creates one.

cd project1/src/main || mkdir -p project1/src/main


the -p command allows you to create any relevant parent directories as well. The && and || control characters can be used together.

Q. How will you check using unix commands that a given class file exists within a jar file?
A. Let us assume that there are a number of jar files sitting under the myjars/lib folder. The following script "search-class-file.sh" sits under myjars.


#!/bin/bash

pattern=$1
shift

for jar in $(find $* -type f -name '*.jar')
do
  match=`jar -tvf $jar | grep $pattern`
  if [ ! -z "$match" ]
  then
    echo "Found in: $jar"
    echo "$match"
  fi
done

The above script can be executed as shown below

$ ./search-class-file.sh Rebalance.class ./lib

$0 --> will be ./search-class-file.sh
$1 --> will be Rebalance.class
After shift, $* will be $1 which is ./lib
-z is a string function
if [ ! -z "$match" ] --> means if $match is not zero length

The output will look like

$ ./search-class-file.sh Rebalance.class ./lib
Found in: ./lib/myapp.jar
  7007 Wed Jul 11 16:44:16 EST 2012 com/myapp/mortgage/MortgageRebalance.class
  5956 Wed Jul 11 16:44:16 EST 2012 com/myapp/bulk/BulkRebalance.class
 15880 Wed Jul 11 16:44:16 EST 2012 com/myapp/model/ModelRebalance.class

similar results can be accomplished with either

$ for i in ./lib/*.jar; do jar -tvf "$i" | grep  -i Rebalance.class; done

or

$ find ./lib -name '*.jar' -o -name '*.war' -type f | xargs -i bash -c "jar -tvf {}| grep Rebalance.class {}"



Q. How will you go about listing all the users' cron jobs?
A. If you are a root user, you can list the cron jobs for all the users as shown below:
for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -l $user; done



Where the /etc/passwd file looks something like

root:x:0:1:Super-User:/:/usr/bin/ksh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
......


If you need to just list your scheduled jobs then try

crontab -l | more



The ouput will look something like:

# job001 - Loads insurance data from ins system into online DB
#job002 - Loads mortgage data from mort system into online DB

# ----------------------------------------------------------------------------------------
0 5 * * 0,1,2,3,4,5,6   . /usr/my_insurance.ksh
10 5 * * 0,1,2,3,4,5,6  . /usr/my_mortgage.ksh



Labels:

Apr 10, 2013

Maven - setting up profiles and tokenizing properties file

Maven is a popular build tools and often you have a requirement to tokenize the name/value pairs in a properties file so that right values can be substituted depending on for which environment (local, dev, test, uat, prod) you are building the package for.


For example, your properties file (portal.properties) under DEV environment might look like


portal.rest.path.accounts=@PORTAL_REST_PATH_ACCOUNT_SERVICE@
portal.rest.path.tax=@PORTAL_REST_PATH_TAX_SERVICE@
portal.logout.url=@PORTAL_LOGOUT_URL@


The values within "@" are tokens that needs to be replaced at build time. The properties files reside under src/main/resources folder as per the Maven structure.Here is a sample DEV resource myapp.properties that sits under /env/dev folder.

PORTAL_REST_PATH_ACCOUNT_SERVICE=http://DEV-1:8080/accounts_service
PORTAL_REST_PATH_TAX_SERVICE=http://DEV-1:8080/tax_service
PORTAL_LOGOUT_URL=https://smlogin-dev.myapp.net/siteminderagent/ssologout/Logout.html


Next step is to configure the pom.xml file in Maven. This is done via the profiles as shown below.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

   ....

    <profiles>
  
  <profile>
   <id>dev-tomcat</id>
   <build>
    <resources>
     <resource>
      <directory>src/main/resources</directory>
      <filtering>true</filtering>
     </resource>
    </resources>
    <filters>
     <filter>env/dev/myapp.properties</filter>
    </filters>
   </build>
  </profile>
 </profiles>
 
 ....
 
</project>


    <build>
  <resources>
   <resource>
    <directory>src/main/resources</directory>
    <filtering>true</filtering>
   </resource>
  </resources>
 </build>


You also need to turn filtering on as shown below in the pom file.



Finally, you can invoke this profile with the -P option in the mvn command.

clean install tomcat:run -Dmaven.tomcat.port=8080 -Pdev-tomcat




There 3 types of Maven profiles.

  • Per Project profile: Defined in the pom.xml file as shown above.
  • Per User profile: Defined in Maven settings.xml file under %USER_HOME%/.m2
  • Global profile: Defined in Maven global settings.xml file under %M2_HOME%/conf/

and these profiles can be activated a number of ways

1. Via command line with -P option as shown above.

2. You can activate it via the Maven settings.xml file as shown below


<settings xmlns="http://maven.apache.org/POM/4.0.0"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
   http://maven.apache.org/xsd/settings-1.0.0.xsd">
   <mirrors>
      <mirror>
         <id>maven.dev.snaponglobal.com</id>
         <name>Internal Artifactory Maven repository</name>
         <url>http://repo1.maven.org/maven2/</url>
         <mirrorOf>*</mirrorOf>
      </mirror>
   </mirrors>
   <activeProfiles>
      <activeProfile>dev-tomcat</activeProfile>
   </activeProfiles>
</settings>
 
All you have to do is type mvn install and the dev-tomcat profile will be kicked off

3. Via environment variables.

<profile>
   <id>dev-tomcat</id>
   <activation>
      <property>
         <name>env</name>
         <value>dev-tomcat</value>
      </property>
   </activation>
</profile>


4. Via operating system (OS)

<profile>
   <id>test</id>
   <activation>
      <os>
         <name>Windows </name>
         <family>Windows</family>
         <arch>x86</arch>
         <version>6.1<version>
      </os>
   </activation>
</profile>

5. Based on a missing file


<profile>
   <id>test</id>
   <activation>
      <file>
         <missing>target/generated-sources/wsdl</missing>
      </file>
   </activation>
</profile>


Q. Can you give an example where you used a profile?
A. Firstly, to tokenize properties as demonstrated in the beginning of the post. Secondly, in local development you set up a datasource for your tomcat server to use as shown below.

    
 <profiles>
  <!-- Used for running tomcat locally to do application testing by substituting 
   tokenized variables from a properties file-->
  <profile>
   <id>dev-tomcat</id>
   <build>
    <resources>
     <resource>
      <directory>src/main/resources</directory>
      <filtering>true</filtering>
     </resource>
    </resources>
    <filters>
     <filter>src/main/filters/myapp.properties</filter>
    </filters>
    <plugins>
     <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>tomcat-maven-plugin</artifactId>
      <configuration>
          <!-- define your datasource here-->
       <contextFile>src/main/resources/tomcat-maven-plugin/context.xml</contextFile>
      </configuration>
     </plugin>
    </plugins>
   </build>
  </profile>
 </profiles>


The context.xml file will look like

<?xml version="1.0" encoding="ISO-8859-1"?>
<Context>
 <Resource
  name="jdbc/myapp_ds"
  type="javax.sql.DataSource"
  driverClassName="com.sybase.jdbc3.jdbc.SybDataSource"
  auth="Container"
  username="aes_service"
  password="service1"
  url="jdbc:sybase:Tds:myserver:5500/my_db?applicationname=myapp"
  initialSize="5"
  maxActive="5"
  maxIdle="1"
  minIdle="0"
  validationQuery="my_validateQuery"
  testOnBorrow="true"
  testOnReturn="false"
  testWhileIdle="false"
  poolPreparedStatements="false"
  removeAbandoned="false"
  logAbandoned="false"
   />
</Context>


Labels: ,

Apr 9, 2013

JavaScript Tutorial -- coding and debugging with FireBug

Step 1: Download and install Mozilla Firefox (latest available version). Also install the add on FireBug. Two other handy add on plugins for the FireBug are YSlow and Cookies.



The small  "Split Screen Screen" button in FireBug as shown above will allow you to have a split screen. You can use the right hand side screen to type in you JavaScript code and execute it. The  left hand side screen can be used for displaying the console.log statements and errors. 

FireBug is a very handy tool to inspect your DOM elements and CSS styles. You can also add break points to your JavaScript for debugging. The FireBug can be turned on and off with the F12 button.Google chrome has similar development tools and you can bring it on with the F12 key.

Step 2: The tutorial below uses bot JavaScript and jQuery. Firsly you need a basic HTML file that downloads the jQuery library and sets the context to write jQuery code. Here is main.html page.

<html lang="en" >
<head>
   
    <title>test app</title>

   <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js?ver=1.9.1"></script>                                                                 
 </head>             

  
<body>

    <h2>Used to load  jQuery library</h2>   

</body>
</html>




Step 3: Here is the sample function that uses both jQuery and JavaScript as shown below. This function compares two objects or values and checks if  both objects or values are same. It returns either true or false as the return value. This function is not fully tested and used for illustration purpose only. Feel free to try different scenarios and fix any issues that arise.

console.clear();

var $ = jQuery;

       function compareObjects(aaaa, bbbb, includeList) {
            if (typeof aaaa !== typeof bbbb) {
                console.log('types are different: ', typeof aaaa, ' !== ', typeof bbbb);
                return false;
            }
            if (typeof aaaa === 'object') {
                for (var i in aaaa) {
                    if (typeof(i) != 'undefined' && $.inArray(i, includeList) && compareObjects(aaaa[i], bbbb[i]) === false) {
                        console.log('children are different [' + i + '] ', aaaa[i], ' !== ', bbbb[i]);
                        return false;
                    }
                }
                // also iterate over bbbb in case this has more properties that don't exist in aaaa
                for (var i in bbbb) {
                    if (typeof(i) != 'undefined' && $.inArray(i, includeList) && compareObjects(bbbb[i], aaaa[i]) === false) {
                       console.log('children are different [' + i + '] ', aaaa[i], ' !== ', bbbb[i]);
                       return false;
                    }
                }
            } else {
                if (aaaa !== bbbb) {
                    console.log('values are different: ', aaaa, ' !== ', bbbb);
                    return false;

                }
            }
            console.log('same [' + i + '] ', aaaa, ' === ', bbbb);
            return true;
        };



compareObjects({a:5}, {a:5}, null)




Step 4: Invoke the  The above code can be typed in the FireBug console as shown below for debugging and execution.




Now, you can follow the above steps to write a JavaScript function and debug the function using FireBug. Also try using the FireBug to inspect HTML and CSS code by clicking on the respective tabs. The Script tab is also used for placing break points and debugging JavaScript.

Labels: ,

Apr 5, 2013

SQL interview questions and answers - scenarios based

Q. How will you go about identifying duplicate records in a table
A. The following SQL query will do the trick

SELECT code, user_name, COUNT(user_name) AS NumOccurrences
  FROM tbl_user
GROUP BY code, user_name
HAVING ( COUNT(user_name) > 1 )


Q. How would you go about deleting the duplicate records?
A. You could do it in a number of steps as shown below.
  • Create a temporary table.
  • Insert the unique records into the temporary table.
  • Delete the records from the original table.
  • Insert the saved single records from the temporary table back to the original table.





Q. How will you go about searching for table and column names that you don't know where they really are? For example, search for a column name to find out in which tables they do exist.
A. You need to query the database system tables. For example, in Sybase, you can query it as shown below.

select a.name, b.name
from sysobjects a, syscolumns b
where a.id = b.id
and b.name like '%split_income%'

Q. How will you go about writing an SQL query for the following scenario?

Valuation table with the following columns portfolioid, accountid, balance, inactiveflag, valuationdttm, and typecd. The portfolio table has columns portfolioid, and portfoliocd.  The account table has columns accountid and accountcd.

Write an SQL query to extract out the accountcd and the corresponding balance for a given portfoliocd and valuationdttm. Please note that there will be multiple balance records for each account, and your query must only extract out a single balance record per account based on the rule 'extract the record with minimum value for typecd'.

A. As you can see in the sample answer below, inner joins are used to join with the relevant tables. A sub query is used  to calculate the min(typecd) to extract the record with minimum value for typecd.



select  acc.accountcd, val.balance
from valuation val
        inner join portfolio pf on pf.portfolioid = val.portfolioid
        inner join account acc on acc.accountid = val.accountid
where pf.portfoliocd = 'AR30'
and   val.valuationdttm = '28 Dec 2012'
and   val.inactiveflag = 'N'
and   acc.inactiveflag = 'N'
and   val.typecd = (select min(val2.typecd) from valuation val2 where val2.valuationdttm = val.valuationdttm and val2.inactiveflag = 'N' and val2.accountid = val.accountid group by accountid)
order by acc.accountcd




Q. If you need to map actual values retrieved from the database to some other value and then sort by these translated values as well, how will you go about accomplishing this in your SQL code?

For example, StatusCd is the column in the Portfolio table,  and it can have the values of New, and Processed. But the SQL query should return a status of 'Excluded' if the ExcludedFlag column is set yes, and 'Sent' if the SentDateTime is not null. iIf none of the above conditions are met, then return the StatusCd as in the database. The sorting needs to be carried out in the order of 'New', 'Processed', 'Sent', and then 'Excluded'.


A. This can be achieved with a switch/case statement. The syntax of switch/case statement can vary among databases. Here is a sample SQL based on Sybase database server.


SELECT PortfolioCd, SentDateTime, ExcludedFlag, StatusCd as ActualStatusCd, 
 case when p.ExcludedFlag = 'Y' then 'Excluded'
 
                    else case when p.SentDateTime is null then p.StatusCd 
                                 else 'Sent'
                   end
            
end as EvaluatedStatusCd
       
 FROM Portfolio p WHERE valuationdttm > '09 Jan 2013'  and InActiveFlag = 'N'
 
 ORDER BY case when p.ExcludedFlag = 'Y' then '4' 
                    else case  when p.SentDateTime is not null then '3'
                           else case when p.StatusCd = 'New' then '1'
                                        when p.StatusCd = 'Processed' then '2'
                                 end
                          end
              end,  

             PortfolioCd   
    


Q. How would you retrieve a date time column converted to string and formatted as dd/mm/yy hh:mm:ss
A. You can use specif functions provided by your database server. These functions are specific to the database server you are using, hence your code cannot be ported to other database servers. Here is an example in Sybase.

SELECT PortfolioCd, 
      convert(char(11), p.SentDateTime, 103) + convert(char(12), p.SentDateTime, 108) as SentDateTime
FROM Portfolio p 
WHERE valuationdttm > '09 Jan 2013'  and InActiveFlag = 'N'

In the above example, the convert function is used to convert the date time field to char. The 103 in Sybase means dd/mm/yy format and and 108 to convert to the time format hh:mm:ss.


Q. How will you go about tuning your SQL and stored procedures?
A. You can use tools like DB Artisan, TOAD, etc to analyze the query plan. The code (in Sybase) below gives you the elapsed time.

Q. How will you go about tuning your SQL and stored procedures?
A. You can use tools like DB Artisan, TOAD, etc to analyze the query plan. The code below gives you the elapsed time.
DECLARE @start datetime, @stop datetime
SET @start = GETDATE()

exec MY_PROC 'AC345', '02 Jan 2013', null, 'N'

SET @stop = GETDATE()
select datediff(ms, @start, @stop)


Proper indexing is key to get good performancee out of your SQL queries.

Q. What are all the different types of indexes?
A. There are three types of indexes

Unique Index: does not allow the field to have duplicate values if the column is unique indexed. Unique index can be applied automatically when primary key is defined.

Clustered Index: reorders the physical order of the table and search based on the key values. Each table can have only one clustered index.

NonClustered Index: does not alter the physical order of the table and maintains logical order of data. Each table can have 999 non-clustered indexes.


You may also like:

Labels:

Apr 4, 2013

Top 3 Core Java Third Party librararies that you use frequently in your Java projects

Q. Can you list a few third party libraries and useful methods that you frequently use in you core Java and enterprise Java development?
A. Firstly, the Apache Commons library has a number of utility methods to simplify your development with a number of proven APIs with utility methods.

Firstly, the toString(  ), equals( .. ), and hashCode( ... ) methods are very frequently used in your domain or value object classes. Here are a few handy Apache commons library methods.


import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
import org.apache.commons.lang.builder.ToStringBuilder;

public Class Customer {

    private String name;

    //...getters and setters

    @Override
    public int hashCode()
    {
        return HashCodeBuilder.reflectionHashCode(this);
    }
    
    @Override
    public boolean equals(Object obj)
    {
        return EqualsBuilder.reflectionEquals(this, obj);
    }
    
    @Override
    public String toString()
    {
        return ToStringBuilder.reflectionToString(this, ToStringStyle.SHORT_PREFIX_STYLE);
    }
 
 
    //...


The FileUtils class has a number of handy file processing methods.

import org.apache.commons.io.FileUtils;

public Class FileUtils {

    //....
    try
        {
            FileUtils.touch(lockFile);
        }
        catch (IOException e)
        {
            logger.error("Error creating a lock file: " + lockFile.getAbsolutePath(), e);
            throw new MyCustomException(e);
        }

    //....


    public void moveFile(File srcFile, File destFile) throws IOException
    {
        FileUtils.moveFile(srcFile, destFile);
    }
 
}


The StringUtils is handy for checking if a given String is blank or empty.

import org.apache.commons.lang.StringUtils;

//....

    if (StringUtils.isEmpty(feedFileMeta.getFilePath())) {
        feedFileMeta.setFilePath(defaultFilePath);
    }


The Apache commons package has so many handy methods.

import org.apache.commons.codec.binary.Base64;

public class ApacheLibBase64Example{

    public static void main(String args[]) throws IOException {
        String origStr = "original String before encoding";

        //encoding  byte array into base 64
        byte[] encoded = Base64.encodeBase64(origStr.getBytes());     
      
        System.out.println("Original String: " + origStr );
        System.out.println("Base64 Encoded String : " + new String(encoded));
      
        //decoding byte array into base64
        byte[] decoded = Base64.decodeBase64(encoded);      
        System.out.println("Base 64 Decoded  String : " + new String(decoded));

    }
}


Package org.springframework.util  has a number of handy utility classes like Assert, ClassUtils, CollectionUtils, FileCopyUtils, FileSystemUtils, ObjectUtils, StringUtils, TYpeUtils (Java 5 Generics support) and ReflectionUtils to name a few.  The Assert utility class that assists in validating arguments. Useful for identifying programmer errors early and clearly at run time. For example,

Assert.notNull(clazz, "The class must not be null");
Assert.isTrue(i > 0, "The value must be greater than zero");


The google Guava library provides a number of precondition checking utilities with the Preconditions class.

checkArgument(i >= 0, "Argument was %s but expected nonnegative", i);
checkArgument(i < j, "Expected i < j, but %s > %s", i, j);
checkNotNull(objA);


Here is an example using the Preconditions class:

@Controller
...
public void order(ProductInfo data) {
    //required in the controller to handle errors and send a sensible error back to the client
    Preconditions.checkArgument(StringUtils.hasText(data.getName()), "Empty name parameter."); //validation
    productService.order(data);
}



public class ProductInfoValidator implements Validator {

    /**
    * This Validator validates just ProductInfo  instances
    */
    public boolean supports(Class clazz) {
        return ProductInfo.class.equals(clazz);
    }

    public void validate(Object obj, Errors e) {
        ValidationUtils.rejectIfEmpty(e, "name", "name.empty");
        ProductInfo p = (ProductInfo) obj;
        if (p.getSku() < 0) {
            e.rejectValue("sku", "negativevalue");
        } else if (p.getSku() > 9999) {
            e.rejectValue("sku", "sku.out.of.range");
        }
    }
}


The Service layer needs to apply validation to promote defensive programming. The design by contract states that your methods should check for pre and post conditions and fail fast if the contract is not met.

@Service
...
public void order(ProductInfo data) {
    //fail fast and defensive programming. aka assertions and design by contract 
    Preconditions.checkArgument(StringUtils.hasText(data.getName()), "Empty name parameter.");
    productDao.order(data);
}

If you have a requirement to work a lot with the Java collections, and performance and memory usage are of utmost importance, then look at the following libraries to work with the collections.
  1. The Trove library provides high speed regular and primitive collections for Java. 
  2. PCJ (Primitive Collections for Java)
  3. Guava (from google)

In a nut shell, the Apache commons from Apache, Spring utils from Spring, and  Guava libraries from Google should make you more productive without having to reinvent the wheel. If you have heavy usage of data structures, then Trove becomes very handy. So, before you roll out your own utility methods, check if they are already provided by these libraries.


Q. What libraries or frameworks do you use to marshaling and unmarshaling  Javas POJOs to CSV, XML, EXCEL, and PDF?
A.  There are a number of libraries available.

BeanIO is an open source Java framework for reading and writing Java beans or plain old java objects (POJO's) from a flat file or stream. BeanIO is ideally suited for batch processing, and currently supports XML, CSV, delimited and fixed length file formats. It is also handy for writing automated tests using Apache Camel where test instructions can be written on a flat file and read via BeanIO. It can also be used to produce CSV and XML download files.

The Maven users can define

    <!-- BeanIO dependency -->
    <dependency>
      <groupId>org.beanio</groupId>
      <artifactId>beanio</artifactId>
      <version>2.0.0</version>
    </dependency>

    <!-- StAX dependencies for JDK 1.5 users -->
    <dependency>
      <groupId>javax.xml</groupId>
      <artifactId>jsr173</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>com.sun.xml.stream</groupId>
      <artifactId>sjsxp</artifactId>
      <version>1.0.2</version>
    </dependency>  


OpenCSV is an alternative for working with CSV files.

Apache POI, JXLS and JExcelApi are handy for working with Excel files in Java.

To create XML, use JAXB, Stax, JiBX, JDOM, XStream, etc. 

To create PDF documents you can use a number of options listed below.
  1. iText and JasperReports.
  2. Apache PDFBox.
  3. BFO.
  4. Converting to XML and then using XSLT to XSL-FO and then render with Apache FOP.

Labels: , ,

Apr 3, 2013

CSS Interview Questions and answers

Modern applications are web driven using HTML, CSS, and JavaScript. HTML is the building block of Web pages. CSS is the language used to make those Web pages pretty. JavaScript is used for dynamically manipulating the DOM elements and their look and feel. Understanding the basics of HTML and CSS will help you build better Web pages. CSS (Cascading Style Sheets) are vary useful and it it really worth understanding its key concepts. This blog explains the key concepts via questions and answers style.

Q. For the HTML snippet below, can you answers the questions listed below.

<div id="myAppDetails">
  <form  class="submit">
   <fieldset>
    <dl>
     <dt>
      <label for="portfolioCd">
       <h4>Portfolio Code</h4>
      </label>
     </dt>
     <dd>
      <input id="portfolioCd" type="text">
     </dd>
    </dl>
    <dl>
     <dt>
      <label for="valuationDate">
       <h4>
        Valuation Date <small>(dd/mm/yyyy)</small>
       </h4>
      </label>
     </dt>
     <dd>
      <input id="valuationDate" type="text">
     </dd>
    </dl>

    <div class="buttons">
     <button class="btn" type="submit">
     <span class="submit">Submit</span> <span class="refresh">Refresh</span>
     </button>
    </div>
    
    </fieldset>
  </form>
 
</div>



Q1. How will you write a CSS selector to hide the submit and refresh buttons?
A1. Here is the CSS

 
<style>
  #myAppDetails form button .submit,
  #myAppDetails form button .refresh {
        display: none;
  }
</style>


In the code snippet above

#myAppDetails :  refres to an element with id  myAppDetails, which is the top "DIV" element.
form :  refers to a child form element under the element with id "myAppDetails".
button : refers to a child button element under the  form element in the hierarchy.
.submit or .refresh : refers to an element with a style class, which has  the value of either "submit" or "refresh" in the hierarchy under the button element.

Finally the the "display:none" will hide the element selected.



Q2. In the above example, if you want to select more specifically the form element that has the style class submit, and a child element button with class  submit or refresh, how will you write the CSS selector to display the selected element?


A2: Here is the CSS snippet.

<style>
    #myAppDetails  form.submit button .submit,
    #myAppDetails  form.refresh button .refresh {
        display: inline;
    }
 
</style>




The above CSS is more specific than the previous one. The previous one selects any form, but this one selects only the form element with a style class of submit.

#myAppDetails : select any element with id "myAppDetails".
form.submit : select an element that has a style class "submit".
button :  a button child element under a form with class "submit"
.submit or .refresh : a child element with a class name either "submit" or "refresh" under a button element.

You can define multiple selectors with a comma.

Q3. What if the button element must immediately follow the form element?
A3.  You can use the symbol ">" to denote immediate. This is not used frequently. Here is the sample.

<style>
    #myAppDetails  form.submit>button .submit,
    #myAppDetails  form.refresh>button .refresh {
        display: inline;
    }
 
</style>


The two core concepts that are difficult to master in CSS are:

  1. CSS Specificity.
  2. CSS Floats


Q4. Why does your CSS rules don't apply to some elements?
A4. You can minimize the time it takes for hunting bugs by understanding how the browsers interpret your code and using developer tools like firebug for Firefox and Google chrome developer tool.

  • Every CSS selector has its place in the specificity hierarchy. If two selectors apply to the same element as demonstrated in the above example, the one with the higher specificity wins. For example,
"#myAppDetails  form.submit button .submit" is more specific than "#myAppDetails form button .submit" because the first one looks for specific form with class value of submit, whereas the second one applies to any form.
  • There are four distinct categories which determine the specificity level of a given selector: inline styles, IDs, classes+attributes and elements. ID selectors have a higher specificity than attribute selectors. You should always try to use IDs to increase the specificity. A class selector beats any number of element selectors. Here is an example:
 <div id="myDivId" style="font-color:blue"  class="aCSSClass" >

In CSS you will have

.aCSSClass {
     font-weight:12;
}

/** CSS comment: attribute css selector **/
[title] {
    color:blue
}

You can calculate the specificity yourself by allocating some weights, but the better approach is to use developer tools like Firebug or Chrome developer tool to calculate it for you and display the effective CSS.

Q5: When will you use a float?
A5: A float is used for laying out elements with CSS in a Web page.

Q6: Where can you apply a float?
A6: You can't float every element on a Web page. To get technical, you can only float block level elements. These are the elements that take up a block of space on the page, like ,
,
    , etc. The other elements known as inline elements like , , , etc can't be floated.


Q7: What are some of the basics you need to keep in mind regarding floats?
A7:

  • When you float an element it becomes a block box. This box can then be shifted to the left or right on the current line. The markup options are float: left, float: right or float: none.
  • You should always set a width on floated items, except if applied directly to an image, which has an implicit width. If no width is set, the results can become unpredictable.
  • The elements can only be floated to the left or right, it is not possible to make an element float in the centre. When we float an element it is shifted to the right or to the left until it reaches the edge of the containing block.
  • Elements following a floated element will wrap around the floated element. If you do not want this to occur, you can apply the “clear” property to these following elements. The four options are clear: left, clear: right, clear: both or clear: none.
  • If the container element is the HTML , the floated div will sit on the left margin of the page. If the container element is itself contained by something else, the floated div will sit on the left margin of the container.
Q8: What tips would you give to someone who is using CSS in their projects?
A8:
  • Use browser based development tools like firebug.
  • For a more professional CSS, use a library like Google bootstrap.
  • Use JavaScript frameworks like jQuery to dynamically manipulate CSS.
  • Understand the core concepts like selectors, specificity, and floats as explained above.

Here is CSS and HTML in action demonstrating float and some of the other  CSS concepts with an example.




Labels: ,