【问题标题】:How to use SAS token in core-site.xml in Azure VM如何在 Azure VM 的 core-site.xml 中使用 SAS 令牌
【发布时间】:2017-05-20 21:14:06
【问题描述】:

我在 Azure 中创建了 Red hat Linux VM,并在其中安装了我的 Java 应用程序。

我已经使用容器创建了存储帐户。

我还在 Azure 门户上为存储帐户创建了 SAS 令牌。以下是令牌:-

SAS 令牌:-

sv=2015-12-11&ss=b&srt=sco&sp=rwdlac&se=2017-01-05T12:04:00Z&st=2017-01-05T11:54:00Z&spr=https&sig=EUWHyuVzv5CRlvSmg6jRuEfa8RnTNi7FLsdfcdspL6EurLMzY%3D

Blob 令牌:-

https://ACCOUNT_NAME.blob.core.windows.net/CONTAINER_NAME?sv=2015-12-11&ss=b&srt=sco&sp=rwdlac&se=2017-01-05T12:04:00Z&st=2017-01-05T11:54:00Z&spr=https&sig=EUWHyuVzv5CRlvSmg6jRuEfa8RnTNi7FLsdfcdspL6EurLMzY%3D

现在我想用上面的令牌访问我的存储容器。

核心站点.xml

<name>fs.azure.sas.CONTAINER_NAME.ACCOUNT_NAME.blob.core.windows.net</name>
<value>sv%3D2015-12-11%26ss%3Db%26srt%3Dsco%26sp%3Drwdlac%26se%3D2017-01-05T12%3A04%3A00Z%26st%3D2017-01-05T11%3A54%3A00Z%26spr%3Dhttps%26sig%3DEUWHyuVzv5CRlvSmg6jRuEfa8RnTNi7FLsdfcdspL6EurLMzY%253D</value>
</property>
 <property>
      <name>fs.defaultFS</name>
      <value>wasb://CONTAINER_NAME@ACCOUNT_NAME.blob.core.windows.net</value>
      <final>true</final>
    </property>

启动应用程序时出现以下错误:-

AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at com.myapp.container.PxBeanContext.startup(PxBeanContext.java:44)
    at com.myapp.jetty.FrontendServer.main(FrontendServer.java:124)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.myapp.library.streaming.files.UploadFileServiceImpl#0' defined in class path resource [system-config.xml]: Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.myapp.library.LibraryStorageImpl#0' defined in class path resource [system-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1455)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:469)
    at com.myapp.container.PxBeanContext.startup(PxBeanContext.java:42)
    ... 1 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.myapp.library.LibraryStorageImpl#0' defined in class path resource [system-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1455)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:274)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1106)
    at com.myapp.container.PxBeanContext.getBean(PxBeanContext.java:156)
    at com.myapp.library.streaming.files.UploadFileServiceImpl.initialize(UploadFileServiceImpl.java:49)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1581)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1522)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)
    ... 11 more
Caused by: java.lang.RuntimeException: org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at com.myapp.hadoop.common.SimpleHdfsFileSystem.initialize(SimpleHdfsFileSystem.java:45)
    at com.myapp.hadoop.hdp2.HadoopDistributionImpl.initializeHdfs(HadoopDistributionImpl.java:63)
    at com.myapp.hadoop.hdp2.UnsecureHadoopDistributionImpl.connectToFileSystem(UnsecureHadoopDistributionImpl.java:22)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.myapp.hadoop.core.DistributionManager$$anon$1.invoke(DistributionManager.scala:135)
    at com.sun.proxy.$Proxy22.connectToFileSystem(Unknown Source)
    at com.myapp.library.LibraryStorageImpl.parseSimpleAuthFileSystem(LibraryStorageImpl.scala:126)
    at com.myapp.library.LibraryStorageImpl.initializeStorageWithPrefix(LibraryStorageImpl.scala:64)
    at com.myapp.library.LibraryStorageImpl.initialize(LibraryStorageImpl.scala:39)
    at com.myapp.library.LibraryStorageImpl.initialize(LibraryStorageImpl.scala:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1581)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1522)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)
    ... 28 more
Caused by: org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.retrieveMetadata(AzureNativeFileSystemStore.java:1930)
    at org.apache.hadoop.fs.azure.NativeAzureFileSystem.mkdirs(NativeAzureFileSystem.java:1868)
    at org.apache.hadoop.fs.azure.NativeAzureFileSystem.mkdirs(NativeAzureFileSystem.java:1834)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1877)
    at com.myapp.hadoop.common.HdfsFileSystem$1.run(HdfsFileSystem.java:98)
    at com.myapp.hadoop.common.HdfsFileSystem$1.run(HdfsFileSystem.java:91)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at com.myapp.hadoop.common.HdfsFileSystem.__initialize(HdfsFileSystem.java:91)
    at com.myapp.hadoop.common.SimpleHdfsFileSystem.initialize(SimpleHdfsFileSystem.java:40)
    ... 47 more
Caused by: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
    at com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:113)
    at org.apache.hadoop.fs.azure.StorageInterfaceImpl$WrappingIterator.hasNext(StorageInterfaceImpl.java:128)
    at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.retrieveMetadata(AzureNativeFileSystemStore.java:1909)
    ... 57 more
Caused by: com.microsoft.azure.storage.StorageException: The specified resource does not exist.
    at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:89)
    at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:307)
    at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:182)
    at com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:109)
    ... 59 more

请帮我解决这个问题。

【问题讨论】:

  • @Walter 请帮我解决这个问题。
  • 您找到解决方案了吗?

标签: authentication cloud azure-storage azure-blob-storage azure-hdinsight


【解决方案1】:

我认为您需要访问密钥而不是 SAS(根据 https://hadoop.apache.org/docs/stable/hadoop-azure/index.html#Configuring_Credentials):

<property>
  <name>fs.azure.account.key.youraccount.blob.core.windows.net</name>
  <value>YOUR ACCESS KEY</value>
</property>

可以在存储帐户级别的门户上找到这些密钥。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2022-01-03
    • 2022-07-19
    • 2017-08-08
    • 1970-01-01
    • 2020-08-07
    相关资源
    最近更新 更多