Best practice for sending complex data

Added by filip szulczewski 187 days ago

Hi,

I'm just wondering what technique would you suggest for sending more complex data like an Array of Objects threw Union?

For example there is an app, let's call it the manager, that is supposed to send an Array of objects, lets say the object has two properties a String value and a uint value. The manager translates the Array to a String, that could be exploded by a preconfigured String stamp.

There are client apps, let's call them clients, that are supposed to get the String, rebuild the Array based on the String, exploded by the String stamp,
and based on the Objects display the String in the uint color.

I would like it to be done threw room attributes, so newly connected clients can display the Array that was already published. Of course when there is a change in the Array I would like all clients to change there display in near real time.

So I guess the question is should the Array be "hosted" in a single Room Attribute, and is there a more efficient way to do a task like this?

Filip Szulczewski


Replies

RE: Best practice for sending complex data - Added by colin moock 187 days ago

hi filip,
future versions of union/reactor will support json-encoded objects for attribute values. when json support is added, you will be able to assign a regular ActionScript 3.0 array as an attribute value, and other clients will be able to retrieve that value as an array.

however, json support might not be in Union 1.0, so you're left with two options:

1) add json support yourself. for an open source library, called as3corelib, that contains the serializer/deserializer you need, see:
http://code.google.com/p/as3corelib/

2) if you don't want to add json support, and your data structure is small and simple, you can write custom code to convert to/from a simple string format. for example, if you have a list of point objects with x and y values, you could send them in a string formatted as:

"x1,y1,x2,y2,...xn,yn" 

for example:

"2,5,33,83,788,53" 

when you read the value, split the string into an array like this:

var list:Array = theValue.split(",");

then loop over the list to create your objects:

var points:Array = [];
for (var i:int = 0; i < list.length; i+=2) {
  points.push({x:list[i], y:list[i+1]});
}

if your data contains user input, such as street addresses, be careful to escape any delimiters in that input before generating your original string attribute value. for example, if your delimiter is a comma, as in the above point example, then convert all commas to "\," before assigning the attribute value. also, don't use the pipe character ("|") because it's reserved internally by Union.

we know this situation isn't ideal, and we are definitely looking forward to improving it in the future, but for now, custom encoding (as shown in the above example) seems to be covering simple cases. more complex cases can be handled with the json lib linked above.

note that whether we implement json or not, and whether you use a json lib or not, either way, you're sending the entire list to every client every time it changes. in situations with large lists, or lists that change very frequently, the brute force approach ("update the entire list every change") might not be viable. we are, therefore, also planning support for server-side multiuser lists, where clients add to, and remove from, lists remotely. clients are then sent changes as updates (deltas), rather than being sent the entire list every time it changes. multiuser lists definitely won't be in 1.0, but they are on our near-term road map for post 1.0 work. remember, though, that if you have a critical need for multiuser server-side lists in a project, you could fairly easily build the functionality yourself in a server-side room module or server module.

kind regards,
colin

RE: Best practice for sending complex data - Added by filip szulczewski 187 days ago

Hi Colin,

thanks for the answer. The future JSON support sounds very interesting. In the meantime i think i'll try the as3corelib way.
I think that Union is a great project. Keep up the good work!

Filip Szulczewski

RE: Best practice for sending complex data - Added by Kenneth Gilpin 118 days ago

Could I just add to this thread that encoding your variables in XML is another option. As I understanding XML traversing is quick in Actionscript - although I'm sure slower than JSON. However it is better than the solution 2, and doesn't involve extra libraries.

kenneth

RE: Best practice for sending complex data - Added by alex winx 45 days ago

Hi all
My opinion is if you like to send complex data you need to forget about ancient string spiting technique and even XML. JSON is something better but if you want also speed and preservation of the Class object(also dynamic objects <new Object()> are slow in compilation and execution) and use the power of Union support of hibernate, you need AMF.(Thx Adobe that decide to reveal its code). Combination with zip or lha or other compression methods you will get even better results.
If you stuck with JSON as3corelib would help you on the client side like
yourObject={now:new Date()}
room.sendModuleMessage("TESTJSON",{data:JSON.encode(yourObject));

but not on the server side.
String input=evt.getMessage().getArg("data");
JSONObject data = new JSONObject(input);

data.getString("now")
In attachement you will find json.jar library. Put it in lib of your union instalation and add it to the startserver.bat like lib\json.jar
I'm bit tightent with scarce JAVE Message interface having 3 not so useful methods(designed only for dynamic Objects) in discussed cases. I hope in future Message would be more customizable in the future so we could send custom structure of the message content and we could write more optimize code instead writing hacks :)).
This made me problems when I tried
room.sendModuleMessage("TESTAMF",{data:AMF.serializring(classObject)); and to use byteArray.compress(), cos

Message content has much more chars above the 64 ASCII array or chars, so it makes big problems in getArg/s parsers of Message obejct to produce something.
The solution was
CurrentDayVO classObject=new CurrentDayVO();
classObject.now = new Date();
room.sendModuleMessage("TESTAMF",{data:Base64.encode(AMFSerializer.serializeToString(classObject)));

In attachement you can find AMFSerializer.as and Base64.as for client side and for server side
you need to include into lib folder of your union instalation BlazeDS flex-messaging-core.jar;flex-messaging-common.jar;(also attached )
and in startserver.bat lib\flex-messaging-core.jar;lib\json.jar;lib\flex-messaging-common.jar; so you can use
AMFSerialize.java like this:
String input=evt.getMessage().getArg("data");
input=Base64decoder.decode(input);
CurrentDayVO output = AMFSerializer.<CurrentDayVO>fromAmf(input);
For sending complex class object you need to make 2 classes one in AS3(CurrentDayVO.as) and one in Java(CurrentDayVO.java).
AS3:
package com.jdftm.vo{
    [Bindable]
    //[RemoteClass(alias="com.jdftm.vo.CurrentDayVO")]
    public class CurrentDayVO{
        private var _now:Date;

        public function get now():Date{
            return _now;
        }

        public function set now(value:Date):void{
            _now=value;
        }
    }
}

JAVA:
package com.jdftm.vo;

import java.util.Date;

    public class CurrentDayVO
    {

        private Date now;

        public CurrentDayVO() {
        }

        public void setNow(Date now) {
            this.now = now;
        }

        public Date getNow() {
            return now;
        }

    }

They need to be in same packages. Dont forget to registerClassAlias("com.jdftm.vo.CurrentDayVO", CurrentDayVO); in you client code so compiler know to preserve Class object strucuture during serialization.
You need to put CurrentDayVO.class (compiled java) into your lib folder of union instalation.
Both files and TestUnion.as and config.xml are included so you can start testing in no time.
TestRoomModule.java is attached too where you can see how to unpack and use JSON or AMF messages sent by the client.
If you find this useful I hope this become Tutorial on the oficial pages. No hard feeling anyway.
I love what you are doing I'll fight for every whole you made. :))
Regards
Alex

AMF.zip (771.5 KB)

RE: Best practice for sending complex data - Added by Kenneth Gilpin 44 days ago

just reading around your post now...

why don't you code AMF support into union platform out of the box? The websites talk about x10 as fast data packaging etc.

kenneth

RE: Best practice for sending complex data - Added by colin moock 44 days ago

alex,
great work! thanks so much for posting that example. if you happen to have a blog/website on which to post your work as a tutorial, we would be very happy to link to it from unionplatform.com.

kenneth,
built-in amf support is definitely on our roadmap. we plan to add JSON support first because it is also widely supported in JavaScript and other languages. we are close to releasing a JavaScript client SDK, so JSON is particularly attractive.

there is no question that complex data support will be very nice to have, but we believe people can already create very interesting content without it, so we plan to ship 1.0 before dealing with things like JSON or AMF. fortunately, it's relatively easy to add complex data support in custom code (we might not have deferred this feature if it was really hard for developers to achieve independently).

colin

RE: Best practice for sending complex data - Added by alex winx 44 days ago

colin
http://winxalex.blogspot.com/2010/07/use-amf-and-json-packaging-in-union.html

kenneth
What I can do is to modify Reactor on the client side to send AMF
and write ServerSideModule that capture all SeverEvents and RoomEvents and prevent its default behavior.
My views were/are that whole communication should use some better packaging then XML. Even with XMLSocket connection <tag>AMF package for example</tag>. I understand dough that XML is hell easier to debug and is cross platform designed.
When AMF is available in PHP,C#,Java and probably someone will write JSAMF. The power of AMF object is they can not just transfer class formatted data but execute functions on server and get results of that action, which with Reflection of C# or Java is really, really powerful.
Regards
Alex

RE: Best practice for sending complex data - Added by Kenneth Gilpin 43 days ago

Alex,

I don't understand all of your post. With the fridge magnets example you can see how you already have access to remote Java server functions. So would I be right in saying the difference between the union platform solution and, say 'BlazeDS' is the complex data support and amf protocol? We have a similar goal as the applications I am writing for depend upon fast communication with a server and using server based Java functions to offload some of the floating point calculations that flash doesn't do so well. Perhaps colin could answer this, but in the delay from: client > serialise data > server > de-serialise data > execute fn > serialise data > client > de-serialise data, isn't the biggest delay the communication between client and server, so improving the efficiency of the serialisation will only have a small effect on the total server lag?

kenneth

RE: Best practice for sending complex data - Added by alex winx 42 days ago

Hi Kenneth
My intention wasn't to compare Union with BlazeDs.
There two things you should be aware:
1st Quality of the package(usable information sent with the package while serialized). In today Object Oriented programming world is very important you can preserve class object structure and data. In most cases time to dispatch package is much bigger then time CPU to deserialize data.
2nd Compression of the package (smaller package size mean less trafic,more users on same bandwidth,less lag)

Currently package fromat look like this (CDATA code omitted):
<U><M>u70</M><L><A>testRoom</A><A>TESTJSON</A><A>data|{now Jul 26 15:05:10 GMT+0200 2010"}</A></L></U>
which is lot of characters and every character adding 1B of size to the package.

It could be (<-package data is Zlib compressed (AMF))
<U><M>u70</M><L><A>testRoom</A><A>TESTJSON</A><A>xÚã6NÎÏÕËJI+ÉÕ+Ë×s.-*JÍ+qI¬ógÏË/ç`t*Zðvk‡</A></L></U>

and even better if message is class object
class Message
{
   public id:int;
   public values:Array;
   public function Message(id,...values)
   {
     this.id=id;
     this.values=new Array(values);
   }
}

and you send
myClassObject=new CurrentDayVO();
myClassObject.now=new Date();
<u>AMF packed(new Message(70,"testRoom",myClassObject))</u>
<u>xÚãRæ÷M-.NLOå-KÌ)M-fÍLáded,I-.    ÊÏÏå6NÎÏÕËJI+ÉÕ+Ë×s.-*JÍ+qI¬ógÏË/ç`t*ZðþUf</u>

I hope now its more clear.
Regards
Alex

RE: Best practice for sending complex data - Added by colin moock 42 days ago

hey kenneth,
with generalized data, compared to most custom string-based serialization code, using amf:

  • reduces the size of the data payload (which saves bandwidth)
  • improves network transfer speed (due to reduced payload size)
  • saves the time required to implement custom serialization functions
  • saves the time required to write code that associates an object with its class (if required)
  • makes serialization/deserialization faster

in many applications, the preceding factors may not be significant. if you are not worried about bandwidth, and your application is already responsive enough for your needs, and your serialization/deserialization functions are relatively simple, then moving to amf won't offer you any benefits. that said, if we were to implement amf natively in union, you'd get the benefits of amf for free, which, at the very least would save you the time required to implement custom serialization functions.

note that in flash, json doesn't have as many benefits as amf. its main purpose is to save you the time required to implement custom serialization functions, and to establish a serialization standard to help applications interoperate better. in the end, json is just a standard for converting a small set of native javascript objects to a string and back (which, of course, is fast if implemented natively, say, at the browser level). depending on the data you are working with, you will likely find that you can write custom serialization code that is faster and produces smaller data payloads than json. and if you're willing to work with byte arrays, you could even improve on amf's performance and size by writing custom serialization code tailored to your specific data.

colin