Support Topics Documentation Slack YouTube Blog

Support for compression encoding


(Roy N.) #1

Is there currently or planned support for compression encoding that can be specified in the HTTP headers? This especially would be useful for the File Service API but would be also beneficial for the Data Service API , etc.

Example:

Accept-Encoding: gzip,deflate,sdch


(Mark Piller) #2

Thanks, Roy. I am adding it to the roadmap.

Regards,
Mark


(Roy N.) #3

Question on whether this is currently implemented?


(Mark Piller) #4

No, we have not gotten to it yet.


(Jean Baptiste Dujardin) #5

Hello,

Is there now implemented?

Thanks


(Anton Govorushkin) #6

Hi, Accept-Encoding header processing support is not implemented

But currently our system works in a way that we compress all data being sent from server side in order to minimize in the volume of data being sent through net. If that is the main concern - it’s already been implemented and works by default without specifying additional headers.

Anton


(Jean Baptiste Dujardin) #7

I would like to request an external API : ParseHub

But when I do a GET request, I have not an JSON response :

How can I do any request on this API?

Thanks


(Anton Govorushkin) #8

You are receiving a response in such a format as you forcibly requesting data in compressed format by specifying above mentioned header. This is more kind of a question to ParseHub - how to get a json response. Try experimenting with cURL with same request but without specifying Accept-Encoding header. If it’s not working - please contact ParseHub support in order to receive a more detailed solution

Best regards,
Anton


(Jean Baptiste Dujardin) #9

Thanks Anton,

When I do the cURL request with “gunzip”, there is no problem, I receive the json response. You can test :

curl -X GET "https://www.parsehub.com/api/v2/runs/tyJ_5hLgFGc9/data?api_key=tZ03ETywobZP" | gunzip

On Codeless, with or without custom header, I have always the same response :frowning:

I think you need to accept compressed response and Accept-Encoding header with the “Network API” component?


(Sergey Chupov) #10

Using codeless in this regard is analogous to using curl as is, i.e. without passing the result to gunzip tool. You can easily check that by issuing the same request with curl but without piping to gunzip:

curl -X GET "https://www.parsehub.com/api/v2/runs/tyJ_5hLgFGc9/data?api_key=tZ03ETywobZP"

…and you get the same zipped response as you do in Codeless.

So in order for this to work you’ll need something an “unzipper” like gunzip in your code, which means you’ll need to implement gunzip with your code. Our service does not provide such functionality, and neither does HTTP itself.

The problem in your case is that the ParseHub endpoint returns a zipped response by default, and to solve this you’ll need to find out a way to ask it for a non-zipped response.

The Accept-Encoding header wouldn’t help in this regard, since it merely informs the remote server about the format your client can handle. It does not implement the actual decoding. You can check it by issuing the same curl request with this header, it still returns the encoded data:

curl -X GET -H Accept-Encoding:gzip "https://www.parsehub.com/api/v2/runs/tyJ_5hLgFGc9/data?api_key=tZ03ETywobZP"