Reusing HDFS storage by several Hadoop installations -

-

is possible reuse hdfs storage 2 or more hadoop installations? or saying in other words, replicate namenode state.

i want build small showcase hadoop cluster (3-5 nodes) , i'd able play around several hadoop distributions (hortonworks , cloudera @ least). have not decided yet, how have them installed simultaneously , seems challenge, i'd decide - possible reuse data stored in hdfs different clusters (physically using same hard disks)?

for simplicity, i'll happy if works combination of hadoop distros , i'm ready lose data @ point, because it's experiment.

update: want use hdfs exclusively 1 chosen hadoop installation @ time. let's 1 day use cloudera, other hortonworks, both use same data in hdfs.

the 1 caveat would need have these on separate machines since not able bind multiple namenodes same port 8020.

having said cloudera , horton works use same hadoop binaries , same configuration options if built yourself. difference in each of management consoles not come base opensource hadoop releases. suggestion configuring single hadoop group , userbase have access same hdfs namenodes / datanodes , jobtrackers, etc.. should able bind namenodes same hdfs file system. have setup each users ssh permissions well.

there limitations though such hdfs supporting exclusive writes only. when first client contacts name-node open file writing, name-node grants lease client create file. when second client tries open same file writing, name-node see lease file granted client, , reject open request second client.

i configure hdfs dirs accordingly in order preserve level of organization.

i did hadoop 0.23 , 2.2.0 in vmware / ubuntu.

lastly take here official hadoop wiki , faq.

good luck, pat


Comments

Post a Comment

Popular posts from this blog

windows - Single EXE to Install Python Standalone Executable for Easy Distribution -

-
i used pyinstaller create standalone exe file pyqt project. did not use single file option because exe created single file option takes long open. therefore, pyinstaller created dist folder contains bunch of files including program.exe file. create setup.exe file place contents of dist folder in program directory , create shortcut in start menu , on desktop. want super simple user. maybe setup.exe files when download program cnet. found inno-setup, looks promising. however, not know if there special considerations because program standalone program create python/pyqt program. thanks! have experience task? there program using task not know about? inno-setup or nsis easiest use. tell them files include , put them , create nice installer you. wrote short tutorial on experiences using innosetup might find helpful: http://www.blog.pythonlibrary.org/2008/08/27/packaging-wxpymail-for-distribution/ note tutorial based around wxpython app wrote, concepts same.
Read more

c# - Access objects in UserControl from MainWindow in WPF -

-
i have simple xaml, how can change text property of textblock in usercontrol1 mainwindow ? <window x:class="refactorxaml.mainwindow" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:refactorxaml="clr-namespace:refactorxaml" title="mainwindow" height="350" width="525"> <grid> <grid.rowdefinitions> <rowdefinition height="100"></rowdefinition> <rowdefinition height="100"></rowdefinition> </grid.rowdefinitions> <grid> <refactorxaml:usercontrol1></refactorxaml:usercontrol1> </grid> <grid grid.row="1"> <button>change text</button> </grid> </grid> </window> <usercon...
Read more

javascript - How to name a jQuery function to make a browser's back button work? -

-
i'm trying write function ajaxyfy web site, works perfect, except function button. that's have now: function loading() { if (typeof history.pushstate !== "undefined") { var historycount = 0; $('.menu-item, .logo').on('click','a',function(e){ e.preventdefault(); $(this).data('clicked', true); if ($(this).is('.logo a')) { var homepage = $(this).attr('href'); function1(homepage); history.pushstate(null, null, homepage); } else if ($(this).is('.projects a')) { var projects = $(this).attr('href'); function2(projects); history.pushstate(null, null, projects); } else { var pages = $(this).attr('href'); function3(pages); history.pushstate(null, null, pages); ...
Read more