Skip to content

path and to.dfs with R launched from two different directories #185

Description

@nturenne

Hi !

i use rmr2 and hadoop as a newbie on windows 7 with hadoop 2.7.1 and R 3.3.2

This are my parameters :

#init env R
Sys.setenv(HADOOP_CMD="D:/Soft/hadoop/bin/hadoop")
Sys.setenv(HADOOP_HOME="D:/Soft/hadoop/")
Sys.setenv("HADOOP_PREFIX"="D:/Soft/hadoop/")
Sys.setenv(HADOOP_STREAMING="D:/Soft/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar")

Sys.setenv(TMP = 'D:\Soft\hadoop')
library(rhdfs)
library(rmr2)
library(ravro)
.jinit()
hdfs.init()

At this point all is ok

if i run with the path on which R is installed all is ok :

getwd()
[1] "C:/Program Files/R/R-3.3.2/bin/x64"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
17/11/16 17:16:08 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib l
ibrary
17/11/16 17:16:08 INFO compress.CodecPool: Got brand-new compressor [.deflate]

BUT if i run on another path there a path problem somewhere and i dont know why

getwd()
[1] "D:/Utilisateurs/turenn"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
Le chemin d'accès spécifié est introuvable.
Warning message:
l'exécution de la commande 'D:/Soft/hadoop/bin/hadoop jar D:/Soft/hadoop/share/h
adoop/tools/lib/hadoop-streaming-2.7.1.jar loadtb /tmp/file30a0419d6021 < /Temp/
RtmpAL2rCq/file30a0240f658e' renvoie un statut 1

why such problem about path when calling the same Rterm from two different directories
did i miss something ?

thank you for help
Nico

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions