AWS Elastic Search Policy, only allow lambda to access Elastic Search










3














I'm working on setting up an ElasticSearch instance on AWS. My goal is to only allow http request from my Lambda function to the ElasticSearch instance. I have created one policy, that gives the 'Lambdaaccess to theElasticSearchinstance. The part I'm struggling with is the inline resource policy forElasticSearchthat will deny all other request that aren't from the 'Lambda.



I have tried setting the ElasticSearch resource policy to Deny all request and then giving my Lambda a role with access to ElasticSearch. While the Lambda is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided. I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.



Lambda Policy




"Version": "2012-10-17",
"Statement": [

"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:ESHttpGet",
"es:CreateElasticsearchDomain",
"es:DescribeElasticsearchDomainConfig",
"es:ListTags",
"es:ESHttpDelete",
"es:GetUpgradeHistory",
"es:AddTags",
"es:ESHttpHead",
"es:RemoveTags",
"es:DeleteElasticsearchDomain",
"es:DescribeElasticsearchDomain",
"es:UpgradeElasticsearchDomain",
"es:ESHttpPost",
"es:UpdateElasticsearchDomainConfig",
"es:GetUpgradeStatus",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
,

"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole"
],
"Resource": "*"

]



ElasticSearch Inline Policy




"Version": "2012-10-17",
"Statement": [

"Effect": "Deny",
"Principal":
"AWS": [
"*"
]
,
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"

]



Lambda Code Using Aws4 and Axios



//process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
function createRecipesIndex(url, resolve, reject)

axios(aws4.sign(
host: process.env.HOST,
method: "PUT",
url: "https://" + process.env.HOST,
path: '/recipes/',
))
.then(response =>
console.log("----- SUCCESS INDEX CREATED -----");
resolve();
)
.catch(error =>
console.log("----- FAILED TO CREATE INDEX -----");
console.log(error);
reject();
);



Note: I have tried creating my index with the inline policy on ElasticSearch set to allow *(all) and removing the aws4 library signature and it works fine. Right now I just want to secure access to this resource.










share|improve this question


























    3














    I'm working on setting up an ElasticSearch instance on AWS. My goal is to only allow http request from my Lambda function to the ElasticSearch instance. I have created one policy, that gives the 'Lambdaaccess to theElasticSearchinstance. The part I'm struggling with is the inline resource policy forElasticSearchthat will deny all other request that aren't from the 'Lambda.



    I have tried setting the ElasticSearch resource policy to Deny all request and then giving my Lambda a role with access to ElasticSearch. While the Lambda is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided. I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.



    Lambda Policy




    "Version": "2012-10-17",
    "Statement": [

    "Sid": "VisualEditor0",
    "Effect": "Allow",
    "Action": [
    "es:ESHttpGet",
    "es:CreateElasticsearchDomain",
    "es:DescribeElasticsearchDomainConfig",
    "es:ListTags",
    "es:ESHttpDelete",
    "es:GetUpgradeHistory",
    "es:AddTags",
    "es:ESHttpHead",
    "es:RemoveTags",
    "es:DeleteElasticsearchDomain",
    "es:DescribeElasticsearchDomain",
    "es:UpgradeElasticsearchDomain",
    "es:ESHttpPost",
    "es:UpdateElasticsearchDomainConfig",
    "es:GetUpgradeStatus",
    "es:ESHttpPut"
    ],
    "Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
    ,

    "Sid": "VisualEditor1",
    "Effect": "Allow",
    "Action": [
    "es:PurchaseReservedElasticsearchInstance",
    "es:DeleteElasticsearchServiceRole"
    ],
    "Resource": "*"

    ]



    ElasticSearch Inline Policy




    "Version": "2012-10-17",
    "Statement": [

    "Effect": "Deny",
    "Principal":
    "AWS": [
    "*"
    ]
    ,
    "Action": [
    "es:*"
    ],
    "Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"

    ]



    Lambda Code Using Aws4 and Axios



    //process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
    function createRecipesIndex(url, resolve, reject)

    axios(aws4.sign(
    host: process.env.HOST,
    method: "PUT",
    url: "https://" + process.env.HOST,
    path: '/recipes/',
    ))
    .then(response =>
    console.log("----- SUCCESS INDEX CREATED -----");
    resolve();
    )
    .catch(error =>
    console.log("----- FAILED TO CREATE INDEX -----");
    console.log(error);
    reject();
    );



    Note: I have tried creating my index with the inline policy on ElasticSearch set to allow *(all) and removing the aws4 library signature and it works fine. Right now I just want to secure access to this resource.










    share|improve this question
























      3












      3








      3







      I'm working on setting up an ElasticSearch instance on AWS. My goal is to only allow http request from my Lambda function to the ElasticSearch instance. I have created one policy, that gives the 'Lambdaaccess to theElasticSearchinstance. The part I'm struggling with is the inline resource policy forElasticSearchthat will deny all other request that aren't from the 'Lambda.



      I have tried setting the ElasticSearch resource policy to Deny all request and then giving my Lambda a role with access to ElasticSearch. While the Lambda is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided. I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.



      Lambda Policy




      "Version": "2012-10-17",
      "Statement": [

      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
      "es:ESHttpGet",
      "es:CreateElasticsearchDomain",
      "es:DescribeElasticsearchDomainConfig",
      "es:ListTags",
      "es:ESHttpDelete",
      "es:GetUpgradeHistory",
      "es:AddTags",
      "es:ESHttpHead",
      "es:RemoveTags",
      "es:DeleteElasticsearchDomain",
      "es:DescribeElasticsearchDomain",
      "es:UpgradeElasticsearchDomain",
      "es:ESHttpPost",
      "es:UpdateElasticsearchDomainConfig",
      "es:GetUpgradeStatus",
      "es:ESHttpPut"
      ],
      "Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
      ,

      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
      "es:PurchaseReservedElasticsearchInstance",
      "es:DeleteElasticsearchServiceRole"
      ],
      "Resource": "*"

      ]



      ElasticSearch Inline Policy




      "Version": "2012-10-17",
      "Statement": [

      "Effect": "Deny",
      "Principal":
      "AWS": [
      "*"
      ]
      ,
      "Action": [
      "es:*"
      ],
      "Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"

      ]



      Lambda Code Using Aws4 and Axios



      //process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
      function createRecipesIndex(url, resolve, reject)

      axios(aws4.sign(
      host: process.env.HOST,
      method: "PUT",
      url: "https://" + process.env.HOST,
      path: '/recipes/',
      ))
      .then(response =>
      console.log("----- SUCCESS INDEX CREATED -----");
      resolve();
      )
      .catch(error =>
      console.log("----- FAILED TO CREATE INDEX -----");
      console.log(error);
      reject();
      );



      Note: I have tried creating my index with the inline policy on ElasticSearch set to allow *(all) and removing the aws4 library signature and it works fine. Right now I just want to secure access to this resource.










      share|improve this question













      I'm working on setting up an ElasticSearch instance on AWS. My goal is to only allow http request from my Lambda function to the ElasticSearch instance. I have created one policy, that gives the 'Lambdaaccess to theElasticSearchinstance. The part I'm struggling with is the inline resource policy forElasticSearchthat will deny all other request that aren't from the 'Lambda.



      I have tried setting the ElasticSearch resource policy to Deny all request and then giving my Lambda a role with access to ElasticSearch. While the Lambda is using that role I am signing my http requests using axios and aws4 but the request are rejected with The request signature we calculated does not match the signature you provided. I don't think the issue is the actual signing of the request but instead the polices I created. If anyone can steer me in the right direction that would really help.



      Lambda Policy




      "Version": "2012-10-17",
      "Statement": [

      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
      "es:ESHttpGet",
      "es:CreateElasticsearchDomain",
      "es:DescribeElasticsearchDomainConfig",
      "es:ListTags",
      "es:ESHttpDelete",
      "es:GetUpgradeHistory",
      "es:AddTags",
      "es:ESHttpHead",
      "es:RemoveTags",
      "es:DeleteElasticsearchDomain",
      "es:DescribeElasticsearchDomain",
      "es:UpgradeElasticsearchDomain",
      "es:ESHttpPost",
      "es:UpdateElasticsearchDomainConfig",
      "es:GetUpgradeStatus",
      "es:ESHttpPut"
      ],
      "Resource": "arn:aws:es:us-east-1:,accountid>:domain/<es-instance>"
      ,

      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
      "es:PurchaseReservedElasticsearchInstance",
      "es:DeleteElasticsearchServiceRole"
      ],
      "Resource": "*"

      ]



      ElasticSearch Inline Policy




      "Version": "2012-10-17",
      "Statement": [

      "Effect": "Deny",
      "Principal":
      "AWS": [
      "*"
      ]
      ,
      "Action": [
      "es:*"
      ],
      "Resource": "arn:aws:es:us-east-1:<account-number>:domain/<es-instance>/*"

      ]



      Lambda Code Using Aws4 and Axios



      //process.env.HOST = search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com
      function createRecipesIndex(url, resolve, reject)

      axios(aws4.sign(
      host: process.env.HOST,
      method: "PUT",
      url: "https://" + process.env.HOST,
      path: '/recipes/',
      ))
      .then(response =>
      console.log("----- SUCCESS INDEX CREATED -----");
      resolve();
      )
      .catch(error =>
      console.log("----- FAILED TO CREATE INDEX -----");
      console.log(error);
      reject();
      );



      Note: I have tried creating my index with the inline policy on ElasticSearch set to allow *(all) and removing the aws4 library signature and it works fine. Right now I just want to secure access to this resource.







      node.js amazon-web-services aws-lambda axios aws-elasticsearch






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 12 '18 at 17:07









      VirtualProdigy

      1,2531529




      1,2531529






















          1 Answer
          1






          active

          oldest

          votes


















          0














          I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy on my ElasticSearch instance. I needed to update it to allow the role that I have given to my Lambda. This was done by getting the role arn from IAM and then creating the below policy to be attached inline on the ElasticSearch instance.



          My second issue was with aws4. the path and the url I set did not match. My path had /xxxx/ while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx. Since the path contained an extra forward slash not found in the url, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D



          Elastic Search Policy




          "Version": "2012-10-17",
          "Statement": [

          "Effect": "Allow",
          "Principal":
          "AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
          ,
          "Action": "es:*",
          "Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"

          ]






          share|improve this answer




















            Your Answer






            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "1"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













            draft saved

            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53266927%2faws-elastic-search-policy-only-allow-lambda-to-access-elastic-search%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy on my ElasticSearch instance. I needed to update it to allow the role that I have given to my Lambda. This was done by getting the role arn from IAM and then creating the below policy to be attached inline on the ElasticSearch instance.



            My second issue was with aws4. the path and the url I set did not match. My path had /xxxx/ while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx. Since the path contained an extra forward slash not found in the url, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D



            Elastic Search Policy




            "Version": "2012-10-17",
            "Statement": [

            "Effect": "Allow",
            "Principal":
            "AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
            ,
            "Action": "es:*",
            "Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"

            ]






            share|improve this answer

























              0














              I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy on my ElasticSearch instance. I needed to update it to allow the role that I have given to my Lambda. This was done by getting the role arn from IAM and then creating the below policy to be attached inline on the ElasticSearch instance.



              My second issue was with aws4. the path and the url I set did not match. My path had /xxxx/ while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx. Since the path contained an extra forward slash not found in the url, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D



              Elastic Search Policy




              "Version": "2012-10-17",
              "Statement": [

              "Effect": "Allow",
              "Principal":
              "AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
              ,
              "Action": "es:*",
              "Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"

              ]






              share|improve this answer























                0












                0








                0






                I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy on my ElasticSearch instance. I needed to update it to allow the role that I have given to my Lambda. This was done by getting the role arn from IAM and then creating the below policy to be attached inline on the ElasticSearch instance.



                My second issue was with aws4. the path and the url I set did not match. My path had /xxxx/ while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx. Since the path contained an extra forward slash not found in the url, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D



                Elastic Search Policy




                "Version": "2012-10-17",
                "Statement": [

                "Effect": "Allow",
                "Principal":
                "AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
                ,
                "Action": "es:*",
                "Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"

                ]






                share|improve this answer












                I found the solution to my issue and it was 2 fold. The first issue was my inline resource policy on my ElasticSearch instance. I needed to update it to allow the role that I have given to my Lambda. This was done by getting the role arn from IAM and then creating the below policy to be attached inline on the ElasticSearch instance.



                My second issue was with aws4. the path and the url I set did not match. My path had /xxxx/ while my url was https://search-<es-instance>-<es-id>.us-east-1.es.amazonaws.com/xxxx. Since the path contained an extra forward slash not found in the url, the signing failed. For anyone else using the library make sure those values are consistent. I hope this helps someone else out in the future :D



                Elastic Search Policy




                "Version": "2012-10-17",
                "Statement": [

                "Effect": "Allow",
                "Principal":
                "AWS": "arn:aws:iam::<account-id>:role/service-role/<role-name>"
                ,
                "Action": "es:*",
                "Resource": "arn:aws:es:us-east-1:<account-id>:domain/<es-instance>/*"

                ]







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 12 '18 at 18:30









                VirtualProdigy

                1,2531529




                1,2531529



























                    draft saved

                    draft discarded
















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53266927%2faws-elastic-search-policy-only-allow-lambda-to-access-elastic-search%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    How to read a connectionString WITH PROVIDER in .NET Core?

                    Node.js Script on GitHub Pages or Amazon S3

                    Museum of Modern and Contemporary Art of Trento and Rovereto