{"componentChunkName":"component---src-components-post-tsx","path":"/circumventing-aws-lambdas-bundle-size-limit","result":{"pageContext":{"searchData":[{"id":"cjjg9ui0t303s010375m42b9w","title":"Turn a Browser Window into a Notepad with This One-Liner","slug":"/turn-a-browser-window-into-a-notepad-with-this-one-liner","avatar":"https://s3.amazonaws.com/contentkit/static/cjjg9ui0t303s010375m42b9w/OYIaJ1KK_400x400(1).png","date":"July 1, 2020"},{"id":"cjiy7xl9o17w701039gvl18a5","title":"Creating a Collaborative Editor with Draftjs for Fun","slug":"/draft-js-collaborative-editor","avatar":"https://s3.amazonaws.com/contentkit/static/cjiy7xl9o17w701039gvl18a5/draft-js.png","date":"June 28, 2020"},{"id":"cjeqgfsdsnmx90167n7j290kt","title":"How To Include SASS In Your React Project","slug":"/sass-react-webpack","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfsdsnmx90167n7j290kt/parcel-bundler.png","date":"May 1, 2020"},{"id":"cjiy4tseq0tnw0103bc5s7sxj","title":"How to Add a Loading Indicator to Material Ui's Component","slug":"/creating-a-material-ui-button-with-spinner-that-reflects-loading-state","avatar":"https://s3.amazonaws.com/contentkit/static/cjiy4tseq0tnw0103bc5s7sxj/material-ui.png","date":"January 28, 2020"},{"id":"cjkq4u7470ihr0157ad175b12","title":"Connecting to AWS Lambda via WebSockets","slug":"/connecting-to-aws-lambda-via-websockets","avatar":"https://s3.amazonaws.com/contentkit/static/cjkq4u7470ihr0157ad175b12/MQTT.js.png","date":"January 12, 2020"},{"id":"cjeqgfnhknnpe0199zbfwwkd8","title":"6 Really Cool APIs to Have Fun With","slug":"/cool-apis-to-have-fun-with","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfnhknnpe0199zbfwwkd8/scale-api.png","date":"January 1, 2020"},{"id":"cjeqgftvknnr30199dvw266qh","title":"Installing Node Canvas in AWS Lambda","slug":"/node-canvas-aws-lambda","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgftvknnr30199dvw266qh/aws-lambda.jpg","date":"December 31, 2019"},{"id":"cjn5fv8kl0pfj0124wwebufms","title":"GoLang Cheatsheet","slug":"/golang-cheatsheet","avatar":"https://s3.amazonaws.com/contentkit/static/cjn5fv8kl0pfj0124wwebufms/golang.png","date":"October 13, 2019"},{"id":"cjn15y8740liw0175u3s707eh","title":"Scheduled Jobs & Work Queues With Postgresql","slug":"/scheduled-jobs-work-queues-with-postgresql","avatar":"https://s3.amazonaws.com/contentkit/static/cjn15y8740liw0175u3s707eh/kxHkAenZ_400x400.jpg","date":"October 8, 2019"},{"id":"cjmym86fh0jy1013398nfzuqu","title":"Creating a Bot to Refill Parking Meters Using AWS Lambda","slug":"/creating-a-bot-to-refill-parking-meters-using-aws-lambda","avatar":"https://s3.amazonaws.com/contentkit/static/cjmym86fh0jy1013398nfzuqu/prwHMlRn_400x400.jpg","date":"October 7, 2019"},{"id":"cjmsaqt6l09le01046qeukpp9","title":"The USPS Tracking API: How To Track Packages","slug":"/usps-tracking-api","avatar":"https://s3.amazonaws.com/contentkit/static/cjmsaqt6l09le01046qeukpp9/usps.png","date":"October 3, 2019"},{"id":"cjeqgfuyenmy20167rycnmiql","title":"Stormpath vs Firebase - A Side-By-Side Comparison","slug":"/stormpath-vs-firebase-a-side-by-side-comparison","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfuyenmy20167rycnmiql/m3cEA33V_400x400.jpg","date":"September 2, 2019"},{"id":"cjeqgfv88nnrt01997wxd4rnl","title":"Sending emails with Firebase Cloud Functions","slug":"/firebase-functions-sending-emails","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfv88nnrt01997wxd4rnl/firebase.jpg","date":"September 2, 2019"},{"id":"cjeqgfqjknnq20199oe3zwi37","title":"Deploying To DigitalOcean From Travis","slug":"/deploying-to-digitalocean-travis","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfqjknnq20199oe3zwi37/digitalocean.jpeg","date":"August 25, 2019"},{"id":"cjkvpbr6p06z50181fg7xvsc3","title":"Circumventing AWS Lambda's Bundle Size Limit","slug":"/circumventing-aws-lambdas-bundle-size-limit","avatar":"https://s3.amazonaws.com/contentkit/static/cjkvpbr6p06z50181fg7xvsc3/b2lWK7c0_400x400.png","date":"August 15, 2019"},{"id":"cjiy45h7m0iba0111f5imt5em","title":"A Quick Way to List All Unicode Characters (Javascript)","slug":"/unicode-characters-javascript","avatar":"https://s3.amazonaws.com/contentkit/static/cjiy45h7m0iba0111f5imt5em/unicode.png","date":"July 29, 2019"},{"id":"ci1rxc41tm8yi7nd156pz9dom","title":"Kibana Rest API","slug":"/kibana-rest-api","avatar":"https://s3.amazonaws.com/contentkit/static/ci1rxc41tm8yi7nd156pz9dom/elastic.png","date":"July 24, 2019"},{"id":"cjeqgfjgennmw0199wha3j6u0","title":"Airtable As A Database For Middleman [Tutorial]","slug":"/airtable-middleman-database","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfjgennmw0199wha3j6u0/airtable-books.png","date":"June 1, 2019"},{"id":"cjeqgfj5qnnmr0199vzd85xuo","title":"Building A Multiplayer Game With Three.Js + WebSockets","slug":"/multiplayer-game-threejs","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfj5qnnmr0199vzd85xuo/three.jpg","date":"June 1, 2019"},{"id":"cjeqgfi8hnnml0199f1jyo1p5","title":"The Best Sketch Plugins","slug":"/the-best-sketch-plugins","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfi8hnnml0199f1jyo1p5/sketch.png","date":"June 1, 2019"},{"id":"d9920hsu8y7365frf2c6fxrx2","title":"Configuring Vault by Hashicorp in AWS EC2","slug":"/configuring-vault-by-hashicorp-in-aws-ec2","avatar":"https://s3.amazonaws.com/contentkit/static/d9920hsu8y7365frf2c6fxrx2/vault.png","date":"April 15, 2019"},{"id":"cjeqgfvsfnns00199mlbff48i","title":"Best Zapier Alternatives","slug":"/best-zapier-alternatives","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfvsfnns00199mlbff48i/zapier-alternative-workato.png","date":"January 31, 2019"},{"id":"cjn3l18390lyt0158i8qxs5dc","title":"Running A Simple Node Web Server On AWS EC2","slug":"/running-a-simple-node-web-server-on-aws-ec2","avatar":"https://s3.amazonaws.com/contentkit/static/cjn3l18390lyt0158i8qxs5dc/aws.png","date":"October 11, 2018"},{"id":"cjeqgfr4snmwz01673m8l4i51","title":"Graph.Cool vs Firebase","slug":"/graphcool-vs-firebase","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfr4snmwz01673m8l4i51/graphcool.png","date":"October 7, 2018"},{"id":"cjklurup00k0201739mflg9sz","title":"Accessing Redis on an Aws EC2 Instance from the Outside\t","slug":"/accessing-redis-on-an-aws-ec2-instance-from-the-outside","avatar":"https://s3.amazonaws.com/contentkit/static/cjklurup00k0201739mflg9sz/logo.jpeg","date":"August 9, 2018"},{"id":"cjkfzvvxt08up0191l38aadq4","title":"Using PDFtk in AWS Lamba","slug":"/using-pdftk-in-aws-lamba","avatar":"https://s3.amazonaws.com/contentkit/static/cjkfzvvxt08up0191l38aadq4/pdftk.png","date":"August 4, 2018"},{"id":"cjeqgfqsgnnq70199l4vc6vim","title":"Configuring WebSockets on Elastic Beanstalk/EC2","slug":"/websockets-aws-elasticbeanstalk-ec2","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfqsgnnq70199l4vc6vim/aws.png","date":"July 15, 2018"},{"id":"cjit96q2yk0is0111qpbmw66s","title":"Getting Started With Gmail API","slug":"/gmail-api-quickstart","avatar":"https://s3.amazonaws.com/contentkit/static/cjit96q2yk0is0111qpbmw66s/gmail.png","date":"June 28, 2018"},{"id":"cjeqgfo23nmwd0167ynqe7xi8","title":"5 Tips For Using NextJs","slug":"/5-tips-for-using-nextjs","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfo23nmwd0167ynqe7xi8/bHjpwZem_400x400.png","date":"June 1, 2018"},{"id":"cjhxixcyhw4ze01035butoukz","title":"React unstable_deferredUpdates","slug":"/react-unstable_deferredupdates","avatar":"https://s3.amazonaws.com/contentkit/static/cjhxixcyhw4ze01035butoukz/OYIaJ1KK_400x400(1).png","date":"June 1, 2018"},{"id":"cjeqgflpnnnoy01993y7aez8a","title":"Simple Web Scraping With Javascript","slug":"/simple-web-scraping","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgflpnnnoy01993y7aez8a/developer-tools-scraping.png","date":"May 1, 2018"},{"id":"cjeqgfk9lnnn101994ll4ursr","title":"Easy: Add Firebase Facebook Login To Your React App","slug":"/firebase-facebook-login-react","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfk9lnnn101994ll4ursr/firebase.png","date":"May 1, 2018"},{"id":"cjeqgfiqrnmtj0167dmz5g5dg","title":"The 5 Best Static Site Web Hosts","slug":"/the-5-best-static-site-web-hosts","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfiqrnmtj0167dmz5g5dg/digital-ocean.png","date":"May 1, 2018"},{"id":"cjeqgfmi2nmvs01670pgebekn","title":"Tips and Tricks For Using NightmareJs","slug":"/nightmare-js","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfmi2nmvs01670pgebekn/nightmare.png","date":"May 1, 2018"},{"id":"cjeqgfkqennn60199707yp1fb","title":"Why You Should Create Your Next React Web App With Firebase","slug":"/firebase-react-tutorial","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfkqennn60199707yp1fb/firebase.jpg","date":"May 1, 2018"},{"id":"cjeqgfpr6nnpx019978qzmaok","title":"How to Flush Data From Heroku Redis","slug":"/heroku-redis-flushall","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfpr6nnpx019978qzmaok/3wgIDj3j_400x400.png","date":"April 1, 2018"},{"id":"cjeqgfm8xnnp30199vxndhkgh","title":"5 Ways To Style React Components","slug":"/5-ways-to-style-react-components","avatar":null,"date":"April 1, 2018"},{"id":"cjeqgflz7nmvm0167yo7wsjg3","title":"Delete Spreadsheet Rows For Google Sheets","slug":"/delete-rows-google-sheets","avatar":null,"date":"April 1, 2018"},{"id":"cjeqgfp84nnps0199dvru09ll","title":"Make An Uptime Monitoring Microservice In Under 50 Lines of Code","slug":"/twilio-uptime-monitoring-node-tutorial","avatar":null,"date":"April 1, 2018"},{"id":"cjeqgfri4nnqd0199zsv6a9fl","title":"Hexo - The Best Static Site Generator? ","slug":"/deploy-a-hexo-blog","avatar":null,"date":"April 1, 2018"},{"id":"cjeqgfoygnnpn0199f31dgk9m","title":"Airtable As A Minimum Viable Database For Your ReactJs Project","slug":"/airtable-reactjs","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfoygnnpn0199f31dgk9m/airtable-screenshot.png","date":"March 2, 2018"},{"id":"cjeqgfn6pnmw0016793f81hpx","title":"The Advanced Guide To ReactJs Checkboxes","slug":"/reactjs-checkboxes","avatar":null,"date":"January 20, 2018"},{"id":"cjeqgfe8znnly01991jo5xkxx","title":"Minimum Viable GraphQL QuickStart","slug":"/minimum-viable-graphql-quickstart","avatar":"https://s3.amazonaws.com/contentkit/static/cjeqgfe8znnly01991jo5xkxx/graphcool-schema.png","date":"January 1, 2018"},{"id":"cjeqgfgegnnmb0199jp3hv2xx","title":"How To Add A Contact Form To Your Ghost Blog","slug":"/ghost-blog-contact-form","avatar":null,"date":"November 1, 2017"},{"id":"cjeqgfmvxnnp90199bdti4r8n","title":"PHP Scraping Tutorial - Scrape Reddit With Goutte","slug":"/php-scraping-tutorial-scrape-reddit-with-goutte","avatar":null,"date":"October 7, 2017"},{"id":"cjeqgfpz1nmwp0167c5j46eij","title":"Cannot read property 'loose' of undefined","slug":"/cannot-read-property-loose-of-undefined","avatar":null,"date":"August 25, 2017"},{"id":"cjeqgftlanmxg0167zipvfzp1","title":"Airtable API Example & Tutorial - Generating Charts","slug":"/airtable-api-example-tutorial","avatar":null,"date":"January 31, 2017"},{"id":"cjeqgfphbnmwk0167ldz2mv2q","title":"React onClick Example and Tutorial","slug":"/react-onclick-example-and-tutorial","avatar":null,"date":"January 2, 2017"},{"id":"cjeqgfq8ynmwu0167z6c51ynp","title":"React Tables - How To Render Tables In ReactJS","slug":"/reactjs-tables","avatar":null,"date":"January 2, 2017"},{"id":"cjeqgfuoznmxu0167ayt2zkc3","title":"How To Add A Class in ReactJS","slug":"/reactjs-add-class","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgfu6gnnrb0199nyrddra0","title":"Fetching Github Blame with the GraphQL API V4","slug":"/github-blame-graphql-api-v4","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgfsmsnnqo0199o2x7dl4x","title":"How To Add Meta Descriptions to Middleman Pages","slug":"/middleman-meta-descriptions","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgfs2wnnqi0199rco2vvz2","title":"Zapier Webhook Post Example & Tutorial","slug":"/zapier-webhook-post-example-tutorial","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgfvhunmy90167t6ouqfmb","title":"Setting Up A Job Queue For A Node App [Tutorial]","slug":"/job-queue-node","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgftbcnnqy0199nc1f92te","title":"NextJs vs Create-React-App - A Side-By-Side Comparison","slug":"/nextjs-vs-create-react-app","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgft2pnnqt01990ty8fmeu","title":"How To Create A Modal In ReactJS [Tutorial]","slug":"/reactjs-modal","avatar":null,"date":"March 14, 2018"},{"id":"cjeqgfugpnnrj0199x7anb33t","title":"How To Use Twilio With ReactJS","slug":"/reactjs-twilio-example-tutorial","avatar":null,"date":"March 14, 2018"}],"post":{"id":"cjkvpbr6p06z50181fg7xvsc3","title":"Circumventing AWS Lambda's Bundle Size Limit","slug":"circumventing-aws-lambdas-bundle-size-limit","published_at":"2019-08-15T04:00:00","created_at":"2018-08-15T22:22:34","excerpt":"Workarounds for the bundle size limit in AWS Lambda.","image":{"id":"10yrhi0c9vdglyx7jkp50yy1x","url":"static/cjkvpbr6p06z50181fg7xvsc3/b2lWK7c0_400x400.png"},"posts_tags":[{"tag":{"id":"3pl7ejawubufz70vjnad","name":"AWS"}}],"date":"August 15, 2019","html":"
As of this writing, the AWS Lambda's bundle size limit is 250MB (uncompressed).
\nThis can be a problem if your function depends on large binaries.
\nThe 250 MB limit can be overcome by fetching the binaries from an S3 bucket and caching them in the /tmp directory. Permissions in Lambda forbid writing to any other directory besides /tmp. The contents of /tmp are retained between executions if the interval between executions is < 10 minutes.
\nThis tactic dramatically increases cold-start latency. However, if your lambda function is a worker processing jobs in a queue the increased cold start latency may be acceptable.
\nMoreover, since AWS Lambda reuses containers, in most cases you will not need to fetch the binary for each invocation. If you have control over the flow or rate of invocations, it may only be necessary to fetch the binary after publishing the function.
\nHere's an example. Let's say you wanted to run LaTeX in AWS Lambda to convert .tex files to PDFs. You might spin up an EC2 container using the same linux AMI as AWS Lambda, compile the binaries, compress them and put them in an S3 bucket. But since they're too large to upload alongside your Lambda function, you'd resort to the above approach.
\nconst got = require('got')\nconst unzipper = require('unzipper')\nconst fs = require('fs')\nconst { promisify } = require('util')\nconst { exec } = require('child_process')\nconst chmod = promisify(fs.chmod)\nconst list = promisify(fs.readdir)\nprocess.env.PATH += ':/tmp/texlive/2018/bin/x86_64-linux/'\nprocess.env.HOME = '/tmp'\nprocess.env.PERL5LIB = '/tmp/texlive/2018/tlpkg/TeXLive/'\nfunction isInstalled () {\n return new Promise((resolve, reject) => {\n exec('pdflatex --version', (error, stdout, stderr) => {\n if (error) resolve(false)\n else resolve(true)\n })\n })\n}\nmodule.exports = async () => {\n const installed = await isInstalled()\n if (!installed) {\n const zip = unzipper.Extract({ path: '/tmp' })\n got.stream('https://s3.amazonaws.com/aws-lambda-binaries/texlive.zip').pipe(zip)\n await new Promise((resolve, reject) => {\n zip.on('close', resolve)\n })\n let files = await list('/tmp/texlive/2018/bin/x86_64-linux/')\n await Promise.all(\n files.map(file =>\n chmod(path.join('/tmp/texlive/2018/bin/x86_64-linux/', file), '0700')\n )\n )\n }\n}
","avatar":"https://s3.amazonaws.com/contentkit/static/cjkvpbr6p06z50181fg7xvsc3/b2lWK7c0_400x400.png"}}}}