open-url branch. I’ll merge it into the master branch soon to make it easier.%open-url.red that provides two functions, async open-url and blocking read-url. This is the old version and the script would be removed soon.%http-scheme.red that provides two schemes: http and https. I’m porting the open-url code to the scheme.read-url is the only function that handles the redirection. It’s understandable, as redirection is technically two or more connections done in series and I’m not sure right now if it can be properly implemented in an async matter. Probably yes but I need to think about it.read http://example.com currently uses the simple-io version of http(s), so you need to use port: open http://example.org wait port close port.close event or available in port/extra/response. On close only the content is returned, in port/extra/response you have everything including the headers.GET, additional headers, or some data), you can use insert port data, but that’s WIP and it’s not fully supported yet. open-url gives you finer control currently.awake function (handler?). Is tere planned to be some kind of template, in other words - working with pure TCP ports, will I have to provide a handler each time?awake function does. For HTTP there’s of course need for some preprocessing (creating a request) and postprocessing (parsing a response).open, insert, close, etc. But that would work for small data only, once you hit some limit, you need to deal with repeated reading in the awake function.awake function, it’s not hard to work with it.open, copy, ... actions, but read used simple-IO directly. This commit enables using read with the new scheme.load completely. HTTP scheme needs to decode at least textual data from binary. But if we have a good MIME decoder, then it can be used both in HTTP scheme and in loadread http://example.org should return just the content, not the headers. HTTP scheme provides headers in port/extra/response/header, so you *can* get the proper type, it’s just a bit unconfortable. Another possibilty is to return converted data on read and bytestream on read/binary. Franky, I’m not sure what is the best way.port/extra/response. Raw data aren’t there, they’re converted on the fly and discarded. I could add them at the cost of higer memory usage.read and read/binary has another problem though. It’s usable only with (blocking) read. If you want to do it manually, with open, copy, close, there’s no way to pass the /binary refinement. One idea is to trigger the conversion with the close action, so until you close the port, you would have raw data, after closethe converted data. But maybe it’s getting too confusing?read can decode it and currently does. And it does it in the close event, because you need whole data before decoding (unlike headers, which are converted to Red on the fly).close to do that - because when I hit close, I know I have all the data.read call and is opaque to the user, why should we care how it works internally?read.read and open/copy/close.close event. So read always returns converted data.read/binary would return the raw byte stream, while read would still do the conversion.read/binary have access to the /binary refinement?read/binary have access to the /binary refinement?read handle redirection? IMO yes.read/binary should imo always return a binary. As for complex protocols as http, not sure. If you give me a .jpg, I simply expect a .jpg, as I can save it to the disk, call externall app to resize it, and I can load it, if I need an image type, no? Also VID accepts image file as a parameter directly view [image %draw-windows.png] read should return binary! also, or if it should convert to a proper type. You prefer binary, but what about HTML? Should it return binary too, or convert it to text? And JSON? Shoult it return binary, raw JSON, or convert it to map!?read %photo.jpg from local file, but can only load it as image! or read/binary it. Wouldn't it therefore be the least suprising option to load http://ima.ge/photo.jpg as image!, to read/binary it as binary! and to fail for unrefined read in the same way as a local read would?/mode refinement to read so you can choose between various working modes. So in one you the .jpg is loaded and in another you leave it. open/binary as in R2?read should handle redirection.read and load. Read is for text and raw binary data. Load is for values. Read/as controls only the text encoding. Load/as controls codec application. For files and URLs, via extension and MIME type respectively, the codec can be determined. But if you're loading a string or binary data, /as is your responsibility. In the future, I imagine we'll see a load wrapper or module that can sniff for formats, but that will be an optional piece, as it's a special case.Read/as controls only the text encoding./as is currently ignored. There’s no support for other codepages than UTF-8 (but that can be fixed easily, as I have support for ten most used encodings and others can be added easily). For HTTP, /as isn’t required, as the encoding is given in the headers. But it makes sense for overriding header encoding if it’s wrong. That happens sometimes.open/binary. That probably would be required to support read/binary in custom schemes (like HTTP). But I’m not against a different mechanism of arg passing. Currently, you can’t redifine read action for your scheme, so until that’s fixed, this is just a theoretical talk. But @qtxie is aware of it and we’ll have a solution soon I hope.read/binary would return raw binary data. I guess we all agree on that. Does that include status line and headers?read would return text (I guess we agree that only the content), but what about other types? If we exclude headers, we lose MIME type info. Other option is to return block!/map!/object!/whatever! with all info but that would complicate some things, like for example instead of writing print read http://www.example.org you would need to write something like print select read http://www.example.org 'data (oh, the horror!). This is the method I prefer, but I’m open to suggestions. We need to pass the metadata somehow.read/binary could also be read/raw. I like it a little more for unprocessed data (if it has no over structures)./options refinement to read so you can specify options like:opt: [ headers: true images: 'undecoded .... ] read/options http://www.slashdot.org opt
read http://example.com should return the content by handling basic MIME types:text/* converts to string! (including text/csv)application/json to map!image/[bmp,png,jpg,gif] to image! (supported image codecs)binary!string! or not and decide what to do.read/binary should always return binary! regardless of the MIME type.read/info can be used (should it make OPTIONS request?), it can return a block with headers and other metadata (should it also return the content as well?)/options is a big and general discussion. We've already gone over it quite a bit in the context of codecs.commands objectsend-to command which sends a command to an address:TEST in the mini console to receive a big chunk of data ending with a constrol string having ! at the end. If it wont happen and length differst between each run, you have encountered the problem.Type of result: string!
Length of result: 413696
Server has answered >
{SCALAMATION!LAST:7246:00000000006082!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000006083!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LA
ST:7246:00000000006084!THIS-SHOULD-START/END-WITH-}
If loadable content:
SCALAMATION!LAST:7246:00000000006082!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000006083!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LAS
T:7246:00000000006084!THIS-SHOULD-START/END-WITH-
Insert a server command > test
Type of result: string!
Length of result: 492728
...
Insert a server command > test
Type of result: string!
Length of result: 24576
Server has answered >
{OULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000000360!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000000361!THIS-SHOULD-START/END
-WITH-AN-ESCALAMATION!LAST:7246:00000000000362!THI}
If loadable content:
OULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000000360!THIS-SHOULD-START/END-WITH-AN-ESCALAMATION!LAST:7246:00000000000361!THIS-SHOULD-START/END-
WITH-AN-ESCALAMATION!LAST:7246:00000000000362!THI
Insert a server command > testType of result: string! Length of result: 32768
rbdt:// scheme that's written exactly for this).%rbdt directory.%rbdt directory.do %rs-server.red>> do %rs-client.red >> write rbdt://localhost [ping] ( ...some debug omitted... ) == [pong]
[pong] as an output, congratulation, your server is running!parse to process input from client so to use your own commands, you need to write your own rules and load them (see lines 225 and 227 in %rs-server.red: load-rules rules %rules-giuseppe.red).%rules-users.red which provides user and session handling (singup, login, sessions, tokens, expiration, ...) and %rules-polls.red which is simple polling application, an example how to use the server.ping command would look as separate ruleset (ping is builtin command):Red[]
name ping ; REQUIRED: name of the ruleset
extern reply ; REQUIRED: `reply` is where the server's reply is stored (this should be simplified later, but now you need to add it)
ping: [ ; rule name - currently not used, but it's intended for auto-documentation purposes
'ping ; the actual command expected from the client
(reply: 'pong) ; reply that the client will recive from the server when the rule is matched
]extern directive described above) though not the protocol format itself. And if it does, it's versioned, so future server versions would understand older clients.parse is case sensitive in block! mode, so the command must be [ping], not [PING].import file! - works as #import in Red, but imported file is bound to ruleset only, not available globallyimport word! from ruleset - because imported files are kept in separate contexts and you may need a function from a different context, there's import. An example is import match-token from users in the %rules-polls.red ruleset, which imports the function match-token that checks if user is logged in based on provided token. rbdtstand for? Is that any existing standard? Or just a new acronym? And if so, could we have some more friendly name here? :-)RBDT#{01} currently%rs-client.red file.red:// then? Could be a little confusing but..rs-server has support for authentication.ping, which should be transferred as plain text instead. But that needs some logic to determine which format to use.READ and I receive an Invalid UTF-8 encoding because of accented characters. I see /as econding refinement but I receive a reserved for future use (or not yet implemented) when used ith latin1 or latin-1. What should I use?wroteand asked for the possible change to write, as using a past tense with events is not consistent anyway. If I remember his reaction properly, it was something along the lines of possible IO architecture revision.