Styjun

Why are sugars in whole fruits not digested the same way sugars in juice are?

Could flaps be raised upward to serve as spoilers / lift dumpers?

How is Sword Coast North governed?

Why have both: BJT and FET transistors on IC output?

Delete the following space

May a hotel provide accommodation for fewer people than booked?

A game of red and black

Is this mechanically safe?

How do I find SFDX CLI default installation folder on Mac?

Password management for kids - what's a good way to start?

Is this popular optical illusion made of a grey-scale image with coloured lines?

Constant Scan spooling

Why don't short runways use ramps for takeoff?

How to innovate in OR

Are some indefinite integrals impossible to compute or just don't exist?

How do I safety check that there is no light in Darkroom / Darkbag?

How to let cacti grow even if no player is near?

Novel - Accidental exploration ship, broadcasts a TV show to let people know what they find

Why did the United States not resort to nuclear weapons in Vietnam?

Is it unprofessional to mention your cover letter and resume are best viewed in Chrome?

Being told my "network" isn't PCI Complaint. I don't even have a server! Do I have to comply?

Adjective for when skills are not improving and I'm depressed about it

Oath of redemption: Does Emmissary of Peace reflect damage taken from Aura of the Guardian?

What are the effects of the elements on 3D printed objects made with "infused" PLA

How can I use a Lambda function to call a Glue function (ETL) when a text file is loaded to an S3 bucket

Is there any way to trigger a AWS Lambda function at the end of an AWS Glue job?Can an AWS Lambda function call anotherAWS lambda function ConnectionError when configured with VPCHow to get latest file-name or file from S3 bucket using event triggered lambdaCall a Lambda Function with AWS Glue"EndpointConnectionError : unable to connect to endpoint https://lambda-xyz/wehwk.comHow to programmatically add files to AWS S3 Bucket?Glue with Lambda function callingPython Boto3 Lambda Upload Temp FileWhy Records in missing in the cloudwatch?Issue while executing a script on ec2 using Lambda

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

2

I am trying to set up a lambda function that activates a Glue function when a .txt file is uploaded to an S3 bucket, I am using python 3.7

So far I have this:

from __future__ import print_function

import json
import boto3
import urllib

print('Loading function')

s3 = boto3.client('s3') 

def lambda_handler(event, context): # handler
 source_bucket = event['Records'][0]['s3']['bucket']['name']
 key = urllib.parse.quote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))
 try:
 # what to put here
 except Exception as e:
 print(e)
 print('Error')
 raise e

But I don't understand how can I call the glue function

python-3.x amazon-s3 aws-lambda aws-glue

share|improve this question

edited Mar 27 at 12:15

Jose

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

add a comment | 

2

I am trying to set up a lambda function that activates a Glue function when a .txt file is uploaded to an S3 bucket, I am using python 3.7

So far I have this:

from __future__ import print_function

import json
import boto3
import urllib

print('Loading function')

s3 = boto3.client('s3') 

def lambda_handler(event, context): # handler
 source_bucket = event['Records'][0]['s3']['bucket']['name']
 key = urllib.parse.quote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))
 try:
 # what to put here
 except Exception as e:
 print(e)
 print('Error')
 raise e

But I don't understand how can I call the glue function

python-3.x amazon-s3 aws-lambda aws-glue

share|improve this question

edited Mar 27 at 12:15

Jose

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

add a comment | 

I am trying to set up a lambda function that activates a Glue function when a .txt file is uploaded to an S3 bucket, I am using python 3.7

So far I have this:

from __future__ import print_function

import json
import boto3
import urllib

print('Loading function')

s3 = boto3.client('s3') 

def lambda_handler(event, context): # handler
 source_bucket = event['Records'][0]['s3']['bucket']['name']
 key = urllib.parse.quote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))
 try:
 # what to put here
 except Exception as e:
 print(e)
 print('Error')
 raise e

But I don't understand how can I call the glue function

python-3.x amazon-s3 aws-lambda aws-glue

share|improve this question

edited Mar 27 at 12:15

Jose

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

from __future__ import print_function

import json
import boto3
import urllib

print('Loading function')

s3 = boto3.client('s3') 

def lambda_handler(event, context): # handler
 source_bucket = event['Records'][0]['s3']['bucket']['name']
 key = urllib.parse.quote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))
 try:
 # what to put here
 except Exception as e:
 print(e)
 print('Error')
 raise e

share|improve this question

edited Mar 27 at 12:15

Jose

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

share|improve this question

edited Mar 27 at 12:15

Jose

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

asked Mar 26 at 22:52

JoseJose

951010 bronze badges

2 Answers
2

active

oldest

votes

1

I manage to do it like this:

from __future__ import print_function
import json
import boto3

client = boto3.client('glue')

def lambda_handler(event, context):
 response = client.start_job_run(JobName = 'GLUE_CODE_NAME')

Later I will post the S3 event

share|improve this answer

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @CodeHunter still I would like if you can check the way I did it, and if the same as yours, Thanks
  
  – Jose
  Mar 28 at 19:59
  

  
  
  
  

add a comment | 

0

You can configure an S3 Event Notification that will trigger this Lambda function when PUT object actions is called on an S3 prefix.

https://docs.aws.amazon.com/AmazonS3/latest/user-guide/enable-event-notifications.html

This lambda function can then trigger the StartJobRun action of Glue API.

https://docs.aws.amazon.com/glue/latest/webapi/API_StartJobRun.html

share|improve this answer

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55367322%2fhow-can-i-use-a-lambda-function-to-call-a-glue-function-etl-when-a-text-file-i%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

1

I manage to do it like this:

from __future__ import print_function
import json
import boto3

client = boto3.client('glue')

def lambda_handler(event, context):
 response = client.start_job_run(JobName = 'GLUE_CODE_NAME')

Later I will post the S3 event

share|improve this answer

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @CodeHunter still I would like if you can check the way I did it, and if the same as yours, Thanks
  
  – Jose
  Mar 28 at 19:59
  

  
  
  
  

add a comment | 

1

I manage to do it like this:

from __future__ import print_function
import json
import boto3

client = boto3.client('glue')

def lambda_handler(event, context):
 response = client.start_job_run(JobName = 'GLUE_CODE_NAME')

Later I will post the S3 event

share|improve this answer

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @CodeHunter still I would like if you can check the way I did it, and if the same as yours, Thanks
  
  – Jose
  Mar 28 at 19:59
  

  
  
  
  

add a comment | 

I manage to do it like this:

from __future__ import print_function
import json
import boto3

client = boto3.client('glue')

def lambda_handler(event, context):
 response = client.start_job_run(JobName = 'GLUE_CODE_NAME')

Later I will post the S3 event

share|improve this answer

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

from __future__ import print_function
import json
import boto3

client = boto3.client('glue')

def lambda_handler(event, context):
 response = client.start_job_run(JobName = 'GLUE_CODE_NAME')

share|improve this answer

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

answered Mar 28 at 19:55

JoseJose

951010 bronze badges

0

You can configure an S3 Event Notification that will trigger this Lambda function when PUT object actions is called on an S3 prefix.

https://docs.aws.amazon.com/AmazonS3/latest/user-guide/enable-event-notifications.html

This lambda function can then trigger the StartJobRun action of Glue API.

https://docs.aws.amazon.com/glue/latest/webapi/API_StartJobRun.html

share|improve this answer

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

add a comment | 

0

You can configure an S3 Event Notification that will trigger this Lambda function when PUT object actions is called on an S3 prefix.

https://docs.aws.amazon.com/AmazonS3/latest/user-guide/enable-event-notifications.html

This lambda function can then trigger the StartJobRun action of Glue API.

https://docs.aws.amazon.com/glue/latest/webapi/API_StartJobRun.html

share|improve this answer

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

add a comment | 

You can configure an S3 Event Notification that will trigger this Lambda function when PUT object actions is called on an S3 prefix.

https://docs.aws.amazon.com/AmazonS3/latest/user-guide/enable-event-notifications.html

This lambda function can then trigger the StartJobRun action of Glue API.

https://docs.aws.amazon.com/glue/latest/webapi/API_StartJobRun.html

share|improve this answer

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

share|improve this answer

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges

answered Mar 27 at 14:18

Ujjwal BhardwajUjjwal Bhardwaj

15011 silver badge99 bronze badges