anujjaiswal.wordpress.com
General Scripting Commands (To be continued) | The results and ramblings of research
https://anujjaiswal.wordpress.com/2013/06/26/general-scripting-commands-to-be-continued
The results and ramblings of research. General Scripting Commands (To be continued). Leave a comment ». As a small startup, we utilize AWS extensively. Most of our files are stored in S3, and sometimes it is painful to download files one by one. However, linux is da bomb. I am listing some of the commands that are especially useful. Please note s3cmd. Must be installed and configured correctly. Creating a list of files to download. S3cmd ls - recursive s3:/ bucket/ grep -i condition files.txt. You are co...
anujjaiswal.wordpress.com
AWS EMR High Performance Bootstrap Actions | The results and ramblings of research
https://anujjaiswal.wordpress.com/2015/02/10/aws-emr-high-performance-bootstrap-action
The results and ramblings of research. AWS EMR High Performance Bootstrap Actions. Leave a comment ». In this post, I describe some EMR bootstrap scripts that are especially helpful in ensuring that the Hadoop clusters run great. As a general rule, I use the c3.xlarge compute or m3.xlarge spot nodes and have been consistently deploying medium sized clusters ( 30 nodes). Aggregated logging: To setup more advanced logging, please use the following configure-hadoop bootstrap. M, mapred.local.dir=/mn...Out O...
anujjaiswal.wordpress.com
Moving all indices in a Database to a new Tablespace | The results and ramblings of research
https://anujjaiswal.wordpress.com/2013/09/10/moving-all-indices-in-a-database-to-a-new-tablespace
The results and ramblings of research. Moving all indices in a Database to a new Tablespace. Leave a comment ». As the title outlines, I need to move all the indices in a database to a new tablespace. Turns out it isn’t too tough. Firstly, we can list all indices in a database using the following command:. Select * from pg catalog.pg indexes where schemaname = 'public' order by tablename. Thus, the simple way to get all indices in database is the command:. September 10, 2013 at 3:53 pm. Get every new pos...
anujjaiswal.wordpress.com
MR and Avro – Passing Avro through Pairs | The results and ramblings of research
https://anujjaiswal.wordpress.com/2015/04/20/mr-and-avro-passing-avro-through-pairs
The results and ramblings of research. MR and Avro – Passing Avro through Pairs. Leave a comment ». The mapper class is as follows:. April 20, 2015 at 5:03 pm. Laquo; AWS EMR High Performance Bootstrap Actions. Leave a Reply Cancel reply. Enter your comment here. Fill in your details below or click an icon to log in:. Address never made public). You are commenting using your WordPress.com account. ( Log Out. You are commenting using your Twitter account. ( Log Out. Notify me of new comments via email.
anujjaiswal.wordpress.com
Stress Testing our AWS Hadoop Deployment – Some more results | The results and ramblings of research
https://anujjaiswal.wordpress.com/2013/05/17/stress-testing-our-aws-hadoop-deployment-some-more-results
The results and ramblings of research. Stress Testing our AWS Hadoop Deployment – Some more results. Test 1 – DFSIO Tests. 1) Write Tests – We first ran the DFSIO write test using. Hadoop jar hadoop-mapreduce-client-jobclient-2.0.0-cdh4.1.2-tests.jar TestDFSIO -write -nrFiles 100 -fileSize 1000. The above command runs a write test which generates 100 output files each size of size 1GB for a total of 100GB. 2) Read Tests – Next we ran the DFSIO read test using. For running the test. 13/05/06 16:23:01 INFO...
anujjaiswal.wordpress.com
Towards building robust hadoop cluster | The results and ramblings of research
https://anujjaiswal.wordpress.com/2013/05/02/towards-building-robust-hadoop-cluster
The results and ramblings of research. Towards building robust hadoop cluster. We have been working on using AWS to build our hadoop infrastructure. AWS is a pretty interesting cloud environment, and I do feel it is a great tool for any startup. However, we have been overwhelmed with numerous issues in getting our cluster running in a robust manner. A colleague (Swatz, and check out here blog) and I have identified some simple parameters that magically help produce a uber cluster:. 6) Logs are written fo...
newgeeks.blogspot.com
Tips&Trick: October 2007
http://newgeeks.blogspot.com/2007_10_01_archive.html
Everyday Tips and Tricks to get technology working. Thursday, October 18, 2007. I have moved my post from blogger to wordpress.here is the link to my new blog posts http:/ ragrawal.wordpress.com. Links to this post. Subscribe to: Posts (Atom). View my complete profile.
geovista.psu.edu
GeoVISTA Grad Student Highlights - Ritesh Agrawal
http://www.geovista.psu.edu/aboutus/members/gradhighlightRA.html
Graduated with his PhD in Geography in the spring of 2010, having worked under Dr. Donna Peuquet. Ritesh currently works as a Research Engineer for AT&T Interactive. In San Francisco, and is one of six people who have been designated as the initial team in the new west-coast AT&T research lab. While Ritesh was a graduate affiliate of the GeoVISTA Center at Penn State, he interned at the Pacific Northwest National Laboratory. He is also the core developer of Memento. More about Ritesh's Thesis Research.
newgeeks.blogspot.com
Tips&Trick: Moved
http://newgeeks.blogspot.com/2007/10/moved.html
Everyday Tips and Tricks to get technology working. Thursday, October 18, 2007. I have moved my post from blogger to wordpress.here is the link to my new blog posts http:/ ragrawal.wordpress.com. 很喜歡你的部落格風格,期待你的更新! . 思想與理論,貴呼先於行動,但行動較思想或理論更高貴 . There was never a night or a problem that could defeat sunrise or hope. . Subscribe to: Post Comments (Atom). View my complete profile.