Support Topics Documentation Slack YouTube Blog

Application exceeded limit


(Paul Barnes) #1

Hi Mark and co

I’m really confused by your billing model.

I have a very simple JS API Service deployed to Backendless. It has one function and, apart from the basic npm install, uses one node package - jsdom. The business logic deployment size is registering as 12mb.

Yet, this simple app breaks all of your available billing plans.

Can you explain your philosophy, because I’m really struggling to understand how any app, let alone one running on cloud99, can be less than 10mb.

What am I missing ?

Thanks
Paul


(Mark Piller) #2

Hi Paul,

Having a single dependency does not necessarily mean the implementation would be “light” or the app is “simple” from the perspective of the amount of resources it takes to run it. The jsdom package has 26 dependencies. Some of these dependencies have also a bunch of other dependencies. For example “jsdom” depends on request, which has 20 dependencies on its own and many of those bring additional packages. Before too long you ended with a huge dependency tree that totals in a very large deployment size.

The documentation includes a section which describes this very scenario. See the “The size of my code is bigger than allowed in the current payment tier” section in the Troubleshooting page of the docs.

Regards,
Mark


(Paul Barnes) #3

Hi Mark

Thanks.

I understand the need to limit resources. Which makes total sense when, as a thing scales, they’ll use more geo fences, database entries, API calls etc. But when a product has stabilised, the code is the code is the code, which doesn’t need to scale in the same way that consumable resources do.

I have followed the advice to reduce the dependencies. However, now when I deploy the app (with a recorded much lower file number), I don’t see the app in the API browser - even though the CLI says it has been successful.

Any thoughts ?

Thanks
Paul


(Mark Piller) #4

Hi Paul,

When you deployed the service, there is a line in the output displaying the summary of the deployment which includes the number of services, event handlers and timers. Did you see what it said there? If you could post the complete output of the npm run deploy command line here, I’ll be happy to take a look.

Regards,
Mark


(Paul Barnes) #5

Hi Mark

Thanks. Here’s the output :

npm run deploy – --verbose --keep-zip

> FlipFind-coderunner-project-template@1.0.0 deploy /Users/paulbarnes/Library/Mobile Documents/com~apple~CloudDocs/Development/htmlParser

> coderunner deploy “–verbose” “–keep-zip”

23:34:26.445 CodeRunner™ Backendless JavaScript CodeRunner v4.7.2

23:34:26.448 Copyright© 2018 Backendless Corp. All rights reserved.

23:34:26.451 IMPORTANT!

The business logic code will be deployed to model “default”.

Any business logic which is already deployed on the server in that model

will be removed and replaced with the code from your current project.

If this is an undesired behavior, stop now and set a different deployment model

either by using the --model argument or changing the model name in coderunner.json.

Would you like to continue? (Y/N)Y

23:34:32.058 Building Model…

23:34:32.070 Reading services/parseHTML.js…

23:34:34.005 ServerCode Model built in 1947ms

23:34:34.006 Services (1):

23:34:34.006 ParseHtmlService (services/parseHTML.js)

23:34:34.043 Preparing app zip file for deployment…

23:34:34.044 File patterns to be included:

23:34:34.044 **

23:34:34.044 !node_modules/**

23:34:34.044 !deploy.zip

23:34:34.044 node_modules/jsdom/**

23:34:34.044 node_modules/abab/**

23:34:34.044 node_modules/acorn/**

23:34:34.044 node_modules/acorn-globals/**

23:34:34.044 node_modules/acorn-walk/**

23:34:34.044 node_modules/array-equal/**

23:34:34.044 node_modules/cssom/**

23:34:34.044 node_modules/cssstyle/**

23:34:34.044 node_modules/data-urls/**

23:34:34.044 node_modules/whatwg-mimetype/**

23:34:34.044 node_modules/whatwg-url/**

23:34:34.044 node_modules/lodash.sortby/**

23:34:34.044 node_modules/tr46/**

23:34:34.044 node_modules/punycode/**

23:34:34.045 node_modules/webidl-conversions/**

23:34:34.045 node_modules/domexception/**

23:34:34.045 node_modules/escodegen/**

23:34:34.045 node_modules/esprima/**

23:34:34.045 node_modules/estraverse/**

23:34:34.045 node_modules/esutils/**

23:34:34.045 node_modules/optionator/**

23:34:34.045 node_modules/deep-is/**

23:34:34.045 node_modules/fast-levenshtein/**

23:34:34.045 node_modules/levn/**

23:34:34.045 node_modules/prelude-ls/**

23:34:34.045 node_modules/type-check/**

23:34:34.045 node_modules/wordwrap/**

23:34:34.045 node_modules/source-map/**

23:34:34.045 node_modules/html-encoding-sniffer/**

23:34:34.045 node_modules/whatwg-encoding/**

23:34:34.045 node_modules/iconv-lite/**

23:34:34.045 node_modules/safer-buffer/**

23:34:34.046 node_modules/nwsapi/**

23:34:34.046 node_modules/parse5/**

23:34:34.046 node_modules/pn/**

23:34:34.046 node_modules/request/**

23:34:34.046 node_modules/aws-sign2/**

23:34:34.046 node_modules/aws4/**

23:34:34.046 node_modules/caseless/**

23:34:34.046 node_modules/combined-stream/**

23:34:34.046 node_modules/delayed-stream/**

23:34:34.046 node_modules/extend/**

23:34:34.046 node_modules/forever-agent/**

23:34:34.046 node_modules/form-data/**

23:34:34.046 node_modules/asynckit/**

23:34:34.046 node_modules/mime-types/**

23:34:34.046 node_modules/mime-db/**

23:34:34.046 node_modules/har-validator/**

23:34:34.046 node_modules/ajv/**

23:34:34.046 node_modules/fast-deep-equal/**

23:34:34.046 node_modules/fast-json-stable-stringify/**

23:34:34.046 node_modules/json-schema-traverse/**

23:34:34.046 node_modules/uri-js/**

23:34:34.046 node_modules/har-schema/**

23:34:34.046 node_modules/http-signature/**

23:34:34.046 node_modules/assert-plus/**

23:34:34.046 node_modules/jsprim/**

23:34:34.047 node_modules/extsprintf/**

23:34:34.047 node_modules/json-schema/**

23:34:34.047 node_modules/verror/**

23:34:34.047 node_modules/core-util-is/**

23:34:34.047 node_modules/sshpk/**

23:34:34.047 node_modules/asn1/**

23:34:34.047 node_modules/bcrypt-pbkdf/**

23:34:34.047 node_modules/tweetnacl/**

23:34:34.047 node_modules/dashdash/**

23:34:34.047 node_modules/ecc-jsbn/**

23:34:34.047 node_modules/jsbn/**

23:34:34.047 node_modules/getpass/**

23:34:34.047 node_modules/is-typedarray/**

23:34:34.047 node_modules/isstream/**

23:34:34.047 node_modules/json-stringify-safe/**

23:34:34.047 node_modules/oauth-sign/**

23:34:34.047 node_modules/performance-now/**

23:34:34.047 node_modules/qs/**

23:34:34.047 node_modules/safe-buffer/**

23:34:34.047 node_modules/tough-cookie/**

23:34:34.047 node_modules/psl/**

23:34:34.047 node_modules/tunnel-agent/**

23:34:34.048 node_modules/uuid/**

23:34:34.048 node_modules/request-promise-native/**

23:34:34.048 node_modules/request-promise-core/**

23:34:34.048 node_modules/lodash/**

23:34:34.048 node_modules/stealthy-require/**

23:34:34.048 node_modules/saxes/**

23:34:34.048 node_modules/xmlchars/**

23:34:34.048 node_modules/symbol-tree/**

23:34:34.048 node_modules/w3c-hr-time/**

23:34:34.048 node_modules/browser-process-hrtime/**

23:34:34.048 node_modules/w3c-xmlserializer/**

23:34:34.048 node_modules/xml-name-validator/**

23:34:34.048 node_modules/ws/**

23:34:34.048 node_modules/async-limiter/**

23:34:34.048 !package.json

23:34:34.048 !coderunner.json

23:34:34.048 !README.md

23:34:34.048 !servercode.iml

23:34:34.048 !servercode.ipr

23:34:34.048 !servercode.iws

23:34:34.048 !node_modules/acorn-walk/**

23:34:34.049 !node_modules/lodash.sortby/**

23:34:34.049 !node_modules/tr46/**

23:34:34.049 !node_modules/punycode/**

23:34:34.049 !node_modules/esprima/**

23:34:34.049 !node_modules/estraverse/**

23:34:34.049 !node_modules/esutils/**

23:34:34.049 !node_modules/optionator/**

23:34:34.049 !node_modules/deep-is/**

23:34:34.049 !node_modules/fast-levenshtein/**

23:34:34.049 !node_modules/levn/**

23:34:34.049 !node_modules/prelude-ls/**

23:34:34.049 !node_modules/type-check/**

23:34:34.049 !node_modules/wordwrap/**

23:34:34.049 !node_modules/source-map/**

23:34:34.049 !node_modules/iconv-lite/**

23:34:34.050 !node_modules/safer-buffer/**

23:34:34.050 !node_modules/aws-sign2/**

23:34:34.050 !node_modules/aws4/**

23:34:34.050 !node_modules/caseless/**

23:34:34.050 !node_modules/combined-stream/**

23:34:34.050 !node_modules/delayed-stream/**

23:34:34.050 !node_modules/extend/**

23:34:34.050 !node_modules/forever-agent/**

23:34:34.050 !node_modules/form-data/**

23:34:34.051 !node_modules/asynckit/**

23:34:34.051 !node_modules/mime-types/**

23:34:34.051 !node_modules/mime-db/**

23:34:34.051 !node_modules/har-validator/**

23:34:34.051 !node_modules/ajv/**

23:34:34.051 !node_modules/fast-deep-equal/**

23:34:34.051 !node_modules/fast-json-stable-stringify/**

23:34:34.051 !node_modules/json-schema-traverse/**

23:34:34.051 !node_modules/uri-js/**

23:34:34.051 !node_modules/har-schema/**

23:34:34.051 !node_modules/http-signature/**

23:34:34.052 !node_modules/assert-plus/**

23:34:34.052 !node_modules/jsprim/**

23:34:34.052 !node_modules/extsprintf/**

23:34:34.052 !node_modules/json-schema/**

23:34:34.052 !node_modules/verror/**

23:34:34.052 !node_modules/core-util-is/**

23:34:34.052 !node_modules/sshpk/**

23:34:34.052 !node_modules/asn1/**

23:34:34.052 !node_modules/bcrypt-pbkdf/**

23:34:34.052 !node_modules/tweetnacl/**

23:34:34.052 !node_modules/dashdash/**

23:34:34.052 !node_modules/ecc-jsbn/**

23:34:34.052 !node_modules/jsbn/**

23:34:34.052 !node_modules/getpass/**

23:34:34.052 !node_modules/is-typedarray/**

23:34:34.052 !node_modules/isstream/**

23:34:34.052 !node_modules/json-stringify-safe/**

23:34:34.052 !node_modules/oauth-sign/**

23:34:34.052 !node_modules/performance-now/**

23:34:34.052 !node_modules/qs/**

23:34:34.052 !node_modules/safe-buffer/**

23:34:34.052 !node_modules/psl/**

23:34:34.052 !node_modules/tunnel-agent/**

23:34:34.052 !node_modules/uuid/**

23:34:34.052 !node_modules/request-promise-core/**

23:34:34.053 !node_modules/lodash/**

23:34:34.053 !node_modules/stealthy-require/**

23:34:34.053 !node_modules/xmlchars/**

23:34:34.053 !node_modules/w3c-hr-time/**

23:34:34.053 !node_modules/browser-process-hrtime/**

23:34:34.053 !node_modules/async-limiter/**

23:34:37.222 1091 files added into deployment archive

23:34:37.361 Deployment archive is saved to /Users/paulbarnes/Library/Mobile Documents/com~apple~CloudDocs/Development/htmlParser/deploy.zip

23:34:37.361 Publishing Model to server

23:34:51.797 Successfully published

Thanks,
Paul


(Mark Piller) #6

Thanks! Could you please let me know your app id?


(Paul Barnes) #7

No Problem. Here you go - A603AF06-3DDD-6D47-FF01-D559C4DF9300


(Vladimir Upirov) #8

Hi Paul
I tried to download you BL project from Coding section and deploy it back and I get error: Cannot find module ‘psl’ (services/parseHTML.js)

17:53:24.471 Building Model…
17:53:24.940 ServerCode Model built in 469ms
17:53:24.940 Errors (1):
17:53:24.940 Cannot find module ‘psl’ (services/parseHTML.js)
17:53:24.941 Error: Nothing to publish

Looks like you have removed the module manually from Coding section and that’s the reason why you can not see the service in the API Service list

could you please send us project you deployed with success result

Also to decrease deployment zip size you can use:
“app”: {
“files”: [
“app/"
],
“exclude”: [
"
/.md",
"**/
.text”,

],
}


(Paul Barnes) #9

Thanks Vladimir

That’s because I started deleting items directly from the file listing in the coding BL in BE.
So I think ideally. I just need to trash everything in that API JS repository and start again.

Is there a way to delete it all ? and reset ?

Thanks
Paul


(Vladimir Upirov) #10

yeah, I believe, you can download the project (or not if you already have it), then remove node_modules directory, remove this line const jsdom = require("jsdom"); from /services/parseHTML.js file and comment some related code inside methods of the service and deploy it, as result all the files will be removed on the server.

Then install the jsdom package from NPM again and play with configuration in coderunner.json file.
I recommend you to exclude all the unnecessary files with the configuration, because if you remove some files in node_modules directory manually they will come back each time when you run “npm i”


(Paul Barnes) #11

Thanks Vladimir

That didn’t seem to worked. I followed your instructions.
When I delete node_modules, comment the code and deploy, I see the skeleton api in the API Services browser.

But when install jsdoc, then uncomment the code, the service deploys successfully, but is not visible in the API Services browser

Regards
Paul


(Vladimir Upirov) #12

Can I try it from my side, I mean try to deploy your project from my PC?
If so, please send your project to us, because I downloaded your project from Coding section and there is no “jsdoc” module in package.json, did you run “npm i jsdoc” without “-S” option?
Thanks.


(Paul Barnes) #13

deploy.zip (9.4 MB)

Hi Vladimir - Please do. I have uploaded the zip file to this reply.

Thanks
Paul


(Vladimir Upirov) #14

thanks, I will try it now


(Vladimir Upirov) #15

I’ve got this

10:31:19.688 Building Model…
10:31:19.699 Reading services/parseHTML.js…
10:31:19.819 Error: Cannot find module ‘psl’
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at Module.require (/Users/valodya/.nvm/versions/node/v8.11.3/lib/node_modules/backendless-coderunner/lib/server-code/api/index.js:385:28)
at require (internal/module.js:11:18)
at Object. (/Users/valodya/Downloads/FlipFind__JS_deploy_coding/node_modules/tough-cookie/lib/pubsuffix-psl.js:32:11)
at Module._compile (module.js:652:30)
at Object.Module._extensions…js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
10:31:19.820 ServerCode Model built in 132ms
10:31:19.821 Errors (1):
10:31:19.821 Cannot find module ‘psl’ (services/parseHTML.js)
10:31:19.822 Error: Error: Nothing to publish


(Vladimir Upirov) #16

could you also share your coderunner.json and package.json files, because they are missed in the deploy.zip archive


(Paul Barnes) #17

Hi

Coderunner :
{
“backendless”: {
“apiServer”: “https://api.backendless.com”,
“msgBroker”: {
“host”: “cl.backendless.com”,
“port”: 6379
}
},
“app”: {
“id”: “A603AF06-3DDD-6D47-FF01-D559C4DF9300”,
“apiKey”: “29C48F8C-4C98-301F-FFD7-2756CF042900”,
“exclude”: [
“package.json”,
“coderunner.json”,
“README.md”,
“servercode.iml”,
“servercode.ipr”,
“servercode.iws”,
“node_modules/acorn-walk/",
"node_modules/lodash.sortby/
”,
“node_modules/tr46/",
"node_modules/punycode/
”,
“node_modules/esprima/",
"node_modules/estraverse/
”,
“node_modules/esutils/",
"node_modules/optionator/
”,
“node_modules/deep-is/",
"node_modules/fast-levenshtein/
”,
“node_modules/levn/",
"node_modules/prelude-ls/
”,
“node_modules/type-check/",
"node_modules/wordwrap/
”,
“node_modules/source-map/",
"node_modules/iconv-lite/
”,
“node_modules/safer-buffer/",
"node_modules/aws-sign2/
”,
“node_modules/aws4/",
"node_modules/caseless/
”,
“node_modules/combined-stream/",
"node_modules/delayed-stream/
”,
“node_modules/extend/",
"node_modules/forever-agent/
”,
“node_modules/form-data/",
"node_modules/asynckit/
”,
“node_modules/mime-types/",
"node_modules/mime-db/
”,
“node_modules/har-validator/",
"node_modules/ajv/
”,
“node_modules/fast-deep-equal/",
"node_modules/fast-json-stable-stringify/
”,
“node_modules/json-schema-traverse/",
"node_modules/uri-js/
”,
“node_modules/har-schema/",
"node_modules/http-signature/
”,
“node_modules/assert-plus/",
"node_modules/jsprim/
”,
“node_modules/extsprintf/",
"node_modules/json-schema/
”,
“node_modules/verror/",
"node_modules/core-util-is/
”,
“node_modules/sshpk/",
"node_modules/asn1/
”,
“node_modules/bcrypt-pbkdf/",
"node_modules/tweetnacl/
”,
“node_modules/dashdash/",
"node_modules/ecc-jsbn/
”,
“node_modules/jsbn/",
"node_modules/getpass/
”,
“node_modules/is-typedarray/",
"node_modules/isstream/
”,
“node_modules/json-stringify-safe/",
"node_modules/oauth-sign/
”,
“node_modules/performance-now/",
"node_modules/qs/
”,
“node_modules/safe-buffer/",
"node_modules/psl/
”,
“node_modules/tunnel-agent/",
"node_modules/uuid/
”,
“node_modules/request-promise-core/",
"node_modules/lodash/
”,
“node_modules/stealthy-require/",
"node_modules/xmlchars/
”,
“node_modules/w3c-hr-time/",
"node_modules/browser-process-hrtime/
”,
“node_modules/async-limiter/**”
]
}
}

Package.json :

{
“name”: “FlipFind-coderunner-project-template”,
“version”: “1.0.1”,
“author”: "paul@projectally.co.uk",
“scripts”: {
“debug”: “coderunner debug”,
“deploy”: “coderunner deploy”
},
“license”: “MIT”,
“devDependencies”: {
“backendless-coderunner”: “^4.6.6”
},
“dependencies”: {
“jsdoc”: “^3.5.5”,
“jsdom”: “^13.0.0”
}
}

Thanks
Paul


(Vladimir Upirov) #18

Found the issue, you totally excluded “node_modules/async-limiter/**”, but some code in your project requires the module, and that’s the reason why the service is not displayed in ApiServices List, if you go to Coding section and open the project you will see the error


You should carefully exclude files because JS Code Runner can’t completely analyze your Business Logic before deploy it, for example: you can require some module inside ServiceMethod and the module will be required only when you call the method in runtime.

So, you need to exclude some “test | examples | readme | any not used files…” files, here is an example for you:
“app”: {
“model”: “default”,
“exclude”: [
“+(git|.idea|lib|test|docs|bin|deploy|debug|scripts)/",
“+(LICENSE|.md|.json|.*)”,
".
”,
/*.test.js",
"
/.iml",
"node_modules/**/+(
.ts|.md|LICENSE)",
“node_modules/es6-promise/",
"node_modules/+(algoliasearch|aws-sdk|node-forge)/dist/
”,
"node_modules/
/+(test|tests)/",
"node_modules/+(backendless|backendless-request)/+(dist|es|src)/

]
}
}