AWS Elastic Search Policy, only allow lambda to access Elastic Search
I'm working on setting up an ElasticSearch
instance on AWS
. My goal is to only allow http request from my Lambda
function to the ElasticSearch
instance. I have created one policy, that gives the 'Lambdaaccess to the
ElasticSearchinstance. The part I'm struggling with is the inline resource policy for
ElasticSearchthat will deny all other request that aren't from the 'Lambda
.
I have tried setting the ElasticSearch
resource policy to Deny
all request and then giving my Lambda
a role with access to ElasticSearch.
While the Lambda
is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided.
I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.
Lambda Policy
"Version": "2012-10-17",
"Statement": [
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:ESHttpGet",
"es:CreateElasticsearchDomain",
"es:DescribeElasticsearchDomainConfig",
"es:ListTags",
"es:ESHttpDelete",
"es:GetUpgradeHistory",
"es:AddTags",
"es:ESHttpHead",
"es:RemoveTags",
"es:DeleteElasticsearchDomain",
"es:DescribeElasticsearchDomain",
"es:UpgradeElasticsearchDomain",
"es:ESHttpPost",
"es:UpdateElasticsearchDomainConfig",
"es:GetUpgradeStatus",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
,
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole"
],
"Resource": "*"
]
ElasticSearch Inline Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Deny",
"Principal":
"AWS": [
"*"
]
,
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"
]
Lambda Code Using Aws4 and Axios
//process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
function createRecipesIndex(url, resolve, reject)
axios(aws4.sign(
host: process.env.HOST,
method: "PUT",
url: "https://" + process.env.HOST,
path: '/recipes/',
))
.then(response =>
console.log("----- SUCCESS INDEX CREATED -----");
resolve();
)
.catch(error =>
console.log("----- FAILED TO CREATE INDEX -----");
console.log(error);
reject();
);
Note: I have tried creating my index with the inline policy on ElasticSearch
set to allow *(all) and removing the aws4
library signature and it works fine. Right now I just want to secure access to this resource.
node.js amazon-web-services aws-lambda axios aws-elasticsearch
add a comment |
I'm working on setting up an ElasticSearch
instance on AWS
. My goal is to only allow http request from my Lambda
function to the ElasticSearch
instance. I have created one policy, that gives the 'Lambdaaccess to the
ElasticSearchinstance. The part I'm struggling with is the inline resource policy for
ElasticSearchthat will deny all other request that aren't from the 'Lambda
.
I have tried setting the ElasticSearch
resource policy to Deny
all request and then giving my Lambda
a role with access to ElasticSearch.
While the Lambda
is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided.
I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.
Lambda Policy
"Version": "2012-10-17",
"Statement": [
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:ESHttpGet",
"es:CreateElasticsearchDomain",
"es:DescribeElasticsearchDomainConfig",
"es:ListTags",
"es:ESHttpDelete",
"es:GetUpgradeHistory",
"es:AddTags",
"es:ESHttpHead",
"es:RemoveTags",
"es:DeleteElasticsearchDomain",
"es:DescribeElasticsearchDomain",
"es:UpgradeElasticsearchDomain",
"es:ESHttpPost",
"es:UpdateElasticsearchDomainConfig",
"es:GetUpgradeStatus",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
,
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole"
],
"Resource": "*"
]
ElasticSearch Inline Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Deny",
"Principal":
"AWS": [
"*"
]
,
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"
]
Lambda Code Using Aws4 and Axios
//process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
function createRecipesIndex(url, resolve, reject)
axios(aws4.sign(
host: process.env.HOST,
method: "PUT",
url: "https://" + process.env.HOST,
path: '/recipes/',
))
.then(response =>
console.log("----- SUCCESS INDEX CREATED -----");
resolve();
)
.catch(error =>
console.log("----- FAILED TO CREATE INDEX -----");
console.log(error);
reject();
);
Note: I have tried creating my index with the inline policy on ElasticSearch
set to allow *(all) and removing the aws4
library signature and it works fine. Right now I just want to secure access to this resource.
node.js amazon-web-services aws-lambda axios aws-elasticsearch
add a comment |
I'm working on setting up an ElasticSearch
instance on AWS
. My goal is to only allow http request from my Lambda
function to the ElasticSearch
instance. I have created one policy, that gives the 'Lambdaaccess to the
ElasticSearchinstance. The part I'm struggling with is the inline resource policy for
ElasticSearchthat will deny all other request that aren't from the 'Lambda
.
I have tried setting the ElasticSearch
resource policy to Deny
all request and then giving my Lambda
a role with access to ElasticSearch.
While the Lambda
is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided.
I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.
Lambda Policy
"Version": "2012-10-17",
"Statement": [
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:ESHttpGet",
"es:CreateElasticsearchDomain",
"es:DescribeElasticsearchDomainConfig",
"es:ListTags",
"es:ESHttpDelete",
"es:GetUpgradeHistory",
"es:AddTags",
"es:ESHttpHead",
"es:RemoveTags",
"es:DeleteElasticsearchDomain",
"es:DescribeElasticsearchDomain",
"es:UpgradeElasticsearchDomain",
"es:ESHttpPost",
"es:UpdateElasticsearchDomainConfig",
"es:GetUpgradeStatus",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
,
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole"
],
"Resource": "*"
]
ElasticSearch Inline Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Deny",
"Principal":
"AWS": [
"*"
]
,
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"
]
Lambda Code Using Aws4 and Axios
//process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
function createRecipesIndex(url, resolve, reject)
axios(aws4.sign(
host: process.env.HOST,
method: "PUT",
url: "https://" + process.env.HOST,
path: '/recipes/',
))
.then(response =>
console.log("----- SUCCESS INDEX CREATED -----");
resolve();
)
.catch(error =>
console.log("----- FAILED TO CREATE INDEX -----");
console.log(error);
reject();
);
Note: I have tried creating my index with the inline policy on ElasticSearch
set to allow *(all) and removing the aws4
library signature and it works fine. Right now I just want to secure access to this resource.
node.js amazon-web-services aws-lambda axios aws-elasticsearch
I'm working on setting up an ElasticSearch
instance on AWS
. My goal is to only allow http request from my Lambda
function to the ElasticSearch
instance. I have created one policy, that gives the 'Lambdaaccess to the
ElasticSearchinstance. The part I'm struggling with is the inline resource policy for
ElasticSearchthat will deny all other request that aren't from the 'Lambda
.
I have tried setting the ElasticSearch
resource policy to Deny
all request and then giving my Lambda
a role with access to ElasticSearch.
While the Lambda
is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided.
I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.
Lambda Policy
"Version": "2012-10-17",
"Statement": [
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:ESHttpGet",
"es:CreateElasticsearchDomain",
"es:DescribeElasticsearchDomainConfig",
"es:ListTags",
"es:ESHttpDelete",
"es:GetUpgradeHistory",
"es:AddTags",
"es:ESHttpHead",
"es:RemoveTags",
"es:DeleteElasticsearchDomain",
"es:DescribeElasticsearchDomain",
"es:UpgradeElasticsearchDomain",
"es:ESHttpPost",
"es:UpdateElasticsearchDomainConfig",
"es:GetUpgradeStatus",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
,
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole"
],
"Resource": "*"
]
ElasticSearch Inline Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Deny",
"Principal":
"AWS": [
"*"
]
,
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"
]
Lambda Code Using Aws4 and Axios
//process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
function createRecipesIndex(url, resolve, reject)
axios(aws4.sign(
host: process.env.HOST,
method: "PUT",
url: "https://" + process.env.HOST,
path: '/recipes/',
))
.then(response =>
console.log("----- SUCCESS INDEX CREATED -----");
resolve();
)
.catch(error =>
console.log("----- FAILED TO CREATE INDEX -----");
console.log(error);
reject();
);
Note: I have tried creating my index with the inline policy on ElasticSearch
set to allow *(all) and removing the aws4
library signature and it works fine. Right now I just want to secure access to this resource.
node.js amazon-web-services aws-lambda axios aws-elasticsearch
node.js amazon-web-services aws-lambda axios aws-elasticsearch
asked Nov 12 '18 at 17:07
VirtualProdigy
1,2531529
1,2531529
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy
on my ElasticSearch
instance. I needed to update it to allow the role that I have given to my Lambda
. This was done by getting the role arn
from IAM
and then creating the below policy to be attached inline on the ElasticSearch
instance.
My second issue was with aws4
. the path
and the url
I set did not match. My path had /xxxx/
while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx
. Since the path
contained an extra forward slash not found in the url
, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D
Elastic Search Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Principal":
"AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
,
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"
]
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53266927%2faws-elastic-search-policy-only-allow-lambda-to-access-elastic-search%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy
on my ElasticSearch
instance. I needed to update it to allow the role that I have given to my Lambda
. This was done by getting the role arn
from IAM
and then creating the below policy to be attached inline on the ElasticSearch
instance.
My second issue was with aws4
. the path
and the url
I set did not match. My path had /xxxx/
while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx
. Since the path
contained an extra forward slash not found in the url
, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D
Elastic Search Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Principal":
"AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
,
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"
]
add a comment |
I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy
on my ElasticSearch
instance. I needed to update it to allow the role that I have given to my Lambda
. This was done by getting the role arn
from IAM
and then creating the below policy to be attached inline on the ElasticSearch
instance.
My second issue was with aws4
. the path
and the url
I set did not match. My path had /xxxx/
while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx
. Since the path
contained an extra forward slash not found in the url
, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D
Elastic Search Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Principal":
"AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
,
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"
]
add a comment |
I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy
on my ElasticSearch
instance. I needed to update it to allow the role that I have given to my Lambda
. This was done by getting the role arn
from IAM
and then creating the below policy to be attached inline on the ElasticSearch
instance.
My second issue was with aws4
. the path
and the url
I set did not match. My path had /xxxx/
while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx
. Since the path
contained an extra forward slash not found in the url
, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D
Elastic Search Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Principal":
"AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
,
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"
]
I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy
on my ElasticSearch
instance. I needed to update it to allow the role that I have given to my Lambda
. This was done by getting the role arn
from IAM
and then creating the below policy to be attached inline on the ElasticSearch
instance.
My second issue was with aws4
. the path
and the url
I set did not match. My path had /xxxx/
while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx
. Since the path
contained an extra forward slash not found in the url
, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D
Elastic Search Policy
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Principal":
"AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
,
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"
]
answered Nov 12 '18 at 18:30
VirtualProdigy
1,2531529
1,2531529
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53266927%2faws-elastic-search-policy-only-allow-lambda-to-access-elastic-search%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown