Cogs Library
Over the next few months I will be releasing a number of open source libraries and tools to the Caché community.
Most of the code has evolved from previous production grade solutions over the years and I am collating it together under a single overarching library package that I am calling Cogs.
I will be releasing it in stages as I complete the documentation for each logical release.
Before each full release I will be making each library / tool available on pre-release evaluation and I am open to anyone wanting to participate and provide early feedback.
The core Cogs library will include...
- A comprehensive set of JSON functions
- A JSON-RPC gateway with RAD development features
- Code generation of JavaScript proxy objects
- Code generation of TypeScript proxy objects with object types and auto completion
- CoffeeTable, an ORM JavaScript client library that is compatible with Caché, PostgreSQL and MongoDB
- Fast and scalable Node.JS connector for Caché (currently experimental)
As well as the above I have been working on a number of tools to make open source development easier and plan to release these under the same Cogs library. These tools include...
- An easy to use unit test library with a web interface
- A markdown documentation editor with live preview that saves documents directly into XDATA source code. These documents live inside Caché and can be previewed from Studio with a single click. The markdown compiler includes mermaid for quick production of diagrams (great for Ensemble developers wanting to document message flows).
- A simple code sync tool for working with Git
Kicking off the first release I have decided to pre-release the Cogs JSON library within the next 10 days. Please get in contact if you are interested in this release.
As a flavour of what to expect, the JSON library has the following features...
- Serialise JSON from Cache objects, with features such as JSON name decorators and calculated JSON value functions
- De-serialisation from JSON to Cache objects, including directly to persistent classes.
- Serialise JSON to arrays and globals
- De-serialise JSON from arrays and globals
- Streamlined SQL resultset serialiser that outputs directly to device
- Special data types for storage of raw JSON
- Handle any combination of type, object, array and list in any combination of nested types
- Special JSON mixer variant to serialise and de-serialise objects of mixed object types.
Example
Extend any class with Cogs.JsonClass...
Class Example.Person Extends (%Persistent, Cogs.JsonClass)
{
Property FirstName As %String;
Property LastName As %String;
/// @JSONNAME=BirthDate
Property DateOfBirth As %Date;
/// @JSONIGNORE
Property Secret As %String;
Property Hobbies As list Of %String;
}take some JSON...
{
"BirthDate":"1970-03-25",
"FirstName":"Sean",
"Hobbies":[
"Photography",
"Walking",
"Football"
],
"LastName":"Connelly"
}parse and save it...
set person=##class(Example.Person).parseJSON(json)
set sc=person.%Save()notice that the library automatically converts date types (as well as boolean, number, null and undefined)
person=<OBJECT REFERENCE>[1@Example.Person]
+----------------- general information ---------------
| oref value: 1
| class name: Example.Person
| reference count: 2
+----------------- attribute values ------------------
| %Concurrency = 1 <Set>
| DateOfBirth = 47200
| FirstName = "Sean"
| LastName = "Connelly"
| Secret = ""
+----------------- swizzled references ---------------
| i%Hobbies = ""
| i%Hobbies(1) = "Photography"
| i%Hobbies(2) = "Walking"
| i%Hobbies(3) = "Football"
| r%Hobbies = "2@%Collection.ListOfDT" <Set>
+-----------------------------------------------------now open the persistent object and call its toJSON() method...
set person=##class(Example.Person).%OpenId(1)
write !,person.toJSON()and its as easy as that...
{"BirthDate":"1970-03-25","FirstName":"Sean","Hobbies":["Photography","Walking","Football"],"LastName":"Connelly"}How fast is it?
The above parseJSON() example will complete in 1/20th of a millisecond whilst its toJSON() will complete in 1/50th of a millisecond
How stable is it?
The core JSON library has evolved from numerous developments and re-writes over the years. Some of these versions underpin numerous hospital applications. The Cogs release is the latest evolution, highly optimised and fully unit tested.
This includes throwing some pretty gnarly JSON at it...
{
"TestAllAsciiChars":" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]Ž‘’“”•–—˜™šžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ",
"TestArrayOfBoolean":{
"ALPHA":true,
"CHARLIE":false,
"DELTA":null,
"ECHO":false,
"FOXTROT":true
},
"TestArrayOfDate":{
"Happy-Horolog :)":"1840-12-31",
"doB":"2126-12-06",
"nully":null
},
"TestArrayOfInteger":{
"A THOUSAND!":1000,
"NINETY NINE":99,
"ONE":1,
"THREE":3,
"TWO":2,
"WORLD DEBT $":59546526326827,
"WORLD WORTH $":241000000000000,
"nully":0
},
"TestArrayOfObject":{
},
"TestArrayOfString":{
"City":"London W1",
"Country":"UK",
"Detective":"Sherlock Holmes",
"District":"Marylebone",
"FOOBAR":"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"Street":"221B Baker Street",
"b4d,Th1ng5!":"\\\"\\\"/\"\b\t\r\f\n",
"not-nully":"",
"nully":null
},
"TestBooleanNo":false,
"TestBooleanYes":true,
"TestDate":"2017-05-31",
"TestEmptyArrayOfBoolean":{
},
"TestEmptyArrayOfDate":{
},
"TestEmptyArrayOfInteger":{
},
"TestEmptyArrayOfString":{
},
"TestEmptyListOfBoolean":[
],
"TestEmptyListOfOfDate":[
],
"TestEmptyListOfOfInteger":[
],
"TestEmptyListOfString":[
],
"TestEscapes":"\\\"\\\"/\"\b\t\r\f\n",
"TestInteger":42,
"TestListOfBoolean":[
true,
false,
false,
null,
true,
false,
true,
true,
false
],
"TestListOfObject":[
],
"TestListOfOfDate":[
"1840-12-31",
"1841-01-10",
"1841-04-10",
"1843-09-27",
null,
"1868-05-18",
"2114-10-16",
"4578-11-27"
],
"TestListOfOfInteger":[
0,
43564356546456,
0.345,
0.2,
0,
-464356.75675,
76567,
3.141592653589793238,
7653475667,
-65753,
45676.56,
-6807.383887076754324
],
"TestListOfString":[
"Sherlock Holmes",
"221B Baker Street",
"Marylebone",
"London W1",
"UK",
null,
"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"\\\"\\\"/\"\b\t\r\f\n",
"0",
"1",
"345345345345345345400"
],
"TestLongNumber":49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,
"TestLongString":"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?",
"TestNotBoolean1":"true",
"TestNotBoolean2":"false",
"TestNullBoolean":null,
"TestNullDate":null,
"TestNullInteger":0,
"TestNullString":null,
"TestNullTimestamp":null,
"TestRawJson":{
"TestAllAsciiChars":" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]Ž‘’“”•–—˜™šžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ",
"TestArrayOfBoolean":{
"ALPHA":true,
"CHARLIE":false,
"DELTA":null,
"ECHO":false,
"FOXTROT":true
},
"TestArrayOfDate":{
"Happy-Horolog :)":"1840-12-31",
"doB":"2126-12-06",
"nully":null
},
"TestArrayOfInteger":{
"A THOUSAND!":1000,
"NINETY NINE":99,
"ONE":1,
"THREE":3,
"TWO":2,
"WORLD DEBT $":59546526326827,
"WORLD WORTH $":241000000000000,
"nully":0
},
"TestArrayOfObject":{
},
"TestArrayOfString":{
"City":"London W1",
"Country":"UK",
"Detective":"Sherlock Holmes",
"District":"Marylebone",
"FOOBAR":"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"Street":"221B Baker Street",
"b4d,Th1ng5!":"\\\"\\\"/\"\b\t\r\f\n",
"not-nully":"",
"nully":null
},
"TestBooleanNo":false,
"TestBooleanYes":true,
"TestDate":"2017-05-31",
"TestEmptyArrayOfBoolean":{
},
"TestEmptyArrayOfDate":{
},
"TestEmptyArrayOfInteger":{
},
"TestEmptyArrayOfString":{
},
"TestEmptyListOfBoolean":[
],
"TestEmptyListOfOfDate":[
],
"TestEmptyListOfOfInteger":[
],
"TestEmptyListOfString":[
],
"TestEscapes":"\\\"\\\"/\"\b\t\r\f\n",
"TestInteger":42,
"TestListOfBoolean":[
true,
false,
false,
null,
true,
false,
true,
true,
false
],
"TestListOfObject":[
],
"TestListOfOfDate":[
"1840-12-31",
"1841-01-10",
"1841-04-10",
"1843-09-27",
null,
"1868-05-18",
"2114-10-16",
"4578-11-27"
],
"TestListOfOfInteger":[
0,
43564356546456,
0.345,
0.2,
0,
-464356.75675,
76567,
3.141592653589793238,
7653475667,
-65753,
45676.56,
-6807.383887076754324
],
"TestListOfString":[
"Sherlock Holmes",
"221B Baker Street",
"Marylebone",
"London W1",
"UK",
null,
"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"\\\"\\\"/\"\b\t\r\f\n",
"0",
"1",
"345345345345345345400"
],
"TestLongNumber":49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,
"TestLongString":"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?",
"TestNotBoolean1":"true",
"TestNotBoolean2":"false",
"TestNullBoolean":null,
"TestNullDate":null,
"TestNullInteger":0,
"TestNullString":null,
"TestNullTimestamp":null,
"TestRawJson":{
"data":{
"TestAllAsciiChars":" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVŽ‘’“”•–—˜™šžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ",
"TestArrayOfBoolean":{
"ALPHA":true,
"CHARLIE":false,
"DELTA":null,
"ECHO":false,
"FOXTROT":true
},
"TestArrayOfDate":{
"Happy-Horolog :)":"1840-12-31",
"doB":"2125-11-27",
"nully":null
},
"TestArrayOfInteger":{
"A THOUSAND!":1000,
"NINETY NINE":99,
"ONE":1,
"THREE":3,
"TWO":2,
"WORLD DEBT $":59546526326827,
"WORLD WORTH $":241000000000000,
"nully":null
},
"TestArrayOfObject":{
},
"TestArrayOfString":{
"City":"London W1",
"Country":"UK",
"Detective":"Sherlock Holmes",
"District":"Marylebone",
"FOOBAR":"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"Street":"221B Baker Street",
"b4d,Th1ng5!":"\\\"\\\"/\"\b\t\r\f\n",
"not-nully":"",
"nully":null
},
"TestBooleanNo":false,
"TestBooleanYes":true,
"TestDate":"2016-05-22",
"TestEmptyArrayOfBoolean":{
},
"TestEmptyArrayOfDate":{
},
"TestEmptyArrayOfInteger":{
},
"TestEmptyArrayOfString":{
},
"TestEmptyListOfBoolean":[
],
"TestEmptyListOfOfDate":[
],
"TestEmptyListOfOfInteger":[
],
"TestEmptyListOfString":[
],
"TestEscapes":"\\\"\\\"/\"\b\t\r\f\n",
"TestInteger":42,
"TestListOfBoolean":[
true,
false,
false,
null,
true,
false,
true,
true,
false
],
"TestListOfObject":[
],
"TestListOfOfDate":[
"1840-12-31",
"1841-01-10",
"1841-04-10",
"1843-09-27",
null,
"1868-05-18",
"2114-10-16",
"4578-11-27"
],
"TestListOfOfInteger":[
"0",
43564356546456,
0.345,
0.2,
null,
-464356.75675,
76567,
3.141592653589793238,
7653475667,
-65753,
45676.56,
-6807.383887076754324
],
"TestListOfString":[
"Sherlock Holmes",
"221B Baker Street",
"Marylebone",
"London W1",
"UK",
null,
"{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}",
"\\\"\\\"/\"\b\t\r\f\n",
"0",
"1",
"345345345345345345400"
],
"TestLongNumber":49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,
"TestLongString":"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?",
"TestNotBoolean1":"true",
"TestNotBoolean2":"false",
"TestNullBoolean":null,
"TestNullDate":null,
"TestNullInteger":null,
"TestNullString":null,
"TestNullTimestamp":null,
"TestRawJson":"",
"TestSingleObject":"",
"TestString":"{[{\\}[/][{[\"FOO\",\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\",\"BAR\"]}]{\\}[/]]}",
"TestStringNotNull":"",
"TestStringOfJSON":"{\"menu\": { \"id\": \"file\", \"value\": \"File\", \"popup\": { \"menuitem\": [ {\"value\": \"New\", \"onclick\": \"CreateNewdoc()\"}, {\"value\": \"Open\", \"onclick\": \"Opendoc()\"}, {\"value\": \"Close\", \"onclick\": \"Closedoc()\"} ] }}}",
"TestTimestamp":"1966-01-27T23:12:02",
"TestTimestampShort":"1966-01-27T00:00:00",
"_id":"FOO"
}
},
"TestSingleObject":"",
"TestString":"{[{\\}[/][{[\"FOO\",\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\",\"BAR\"]}]{\\}[/]]}",
"TestStringNotNull":"",
"TestStringOfJSON":"{\"menu\": { \"id\": \"file\", \"value\": \"File\", \"popup\": { \"menuitem\": [ {\"value\": \"New\", \"onclick\": \"CreateNewdoc()\"}, {\"value\": \"Open\", \"onclick\": \"Opendoc()\"}, {\"value\": \"Close\", \"onclick\": \"Closedoc()\"} ] }}}",
"TestTime":"01:14:04",
"TestTimestamp":"1966-01-27T23:12:02",
"TestTimestampShort":"1966-01-27T00:00:00",
"TestZero":0,
"ZTestJsonMethod":"LOREM IPSUM DOLOR SIT AMET, CONSECTETUR ADIPISCING ELIT, SED DO EIUSMOD TEMPOR INCIDIDUNT UT LABORE ET DOLORE MAGNA ALIQUA. UT ENIM AD MINIM VENIAM, QUIS NOSTRUD EXERCITATION ULLAMCO LABORIS NISI UT ALIQUIP EX EA COMMODO CONSEQUAT. DUIS AUTE IRURE DOLOR IN REPREHENDERIT IN VOLUPTATE VELIT ESSE CILLUM DOLORE EU FUGIAT NULLA PARIATUR. EXCEPTEUR SINT OCCAECAT CUPIDATAT NON PROIDENT, SUNT IN CULPA QUI OFFICIA DESERUNT MOLLIT ANIM ID EST LABORUM. SED UT PERSPICIATIS UNDE OMNIS ISTE NATUS ERROR SIT VOLUPTATEM ACCUSANTIUM DOLOREMQUE LAUDANTIUM, TOTAM REM APERIAM, EAQUE IPSA QUAE AB ILLO INVENTORE VERITATIS ET QUASI ARCHITECTO BEATAE VITAE DICTA SUNT EXPLICABO. NEMO ENIM IPSAM VOLUPTATEM QUIA VOLUPTAS SIT ASPERNATUR AUT ODIT AUT FUGIT, SED QUIA CONSEQUUNTUR MAGNI DOLORES EOS QUI RATIONE VOLUPTATEM SEQUI NESCIUNT. NEQUE PORRO QUISQUAM EST, QUI DOLOREM IPSUM QUIA DOLOR SIT AMET, CONSECTETUR, ADIPISCI VELIT, SED QUIA NON NUMQUAM EIUS MODI TEMPORA INCIDUNT UT LABORE ET DOLORE MAGNAM ALIQUAM QUAERAT VOLUPTATEM. UT ENIM AD MINIMA VENIAM, QUIS NOSTRUM EXERCITATIONEM ULLAM CORPORIS SUSCIPIT LABORIOSAM, NISI UT ALIQUID EX EA COMMODI CONSEQUATUR? QUIS AUTEM VEL EUM IURE REPREHENDERIT QUI IN EA VOLUPTATE VELIT ESSE QUAM NIHIL MOLESTIAE CONSEQUATUR, VEL ILLUM QUI DOLOREM EUM FUGIAT QUO VOLUPTAS NULLA PARIATUR?",
"_id":"FOO"
},
"TestSingleObject":"",
"TestString":"{[{\\}[/][{[\"FOO\",\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\",\"BAR\"]}]{\\}[/]]}",
"TestStringNotNull":"",
"TestStringOfJSON":"{\"menu\": { \"id\": \"file\", \"value\": \"File\", \"popup\": { \"menuitem\": [ {\"value\": \"New\", \"onclick\": \"CreateNewdoc()\"}, {\"value\": \"Open\", \"onclick\": \"Opendoc()\"}, {\"value\": \"Close\", \"onclick\": \"Closedoc()\"} ] }}}",
"TestTime":"01:14:04",
"TestTimestamp":"1966-01-27T23:12:02",
"TestTimestampShort":"1966-01-27T00:00:00",
"TestZero":0,
"ZTestJsonMethod":"LOREM IPSUM DOLOR SIT AMET, CONSECTETUR ADIPISCING ELIT, SED DO EIUSMOD TEMPOR INCIDIDUNT UT LABORE ET DOLORE MAGNA ALIQUA. UT ENIM AD MINIM VENIAM, QUIS NOSTRUD EXERCITATION ULLAMCO LABORIS NISI UT ALIQUIP EX EA COMMODO CONSEQUAT. DUIS AUTE IRURE DOLOR IN REPREHENDERIT IN VOLUPTATE VELIT ESSE CILLUM DOLORE EU FUGIAT NULLA PARIATUR. EXCEPTEUR SINT OCCAECAT CUPIDATAT NON PROIDENT, SUNT IN CULPA QUI OFFICIA DESERUNT MOLLIT ANIM ID EST LABORUM. SED UT PERSPICIATIS UNDE OMNIS ISTE NATUS ERROR SIT VOLUPTATEM ACCUSANTIUM DOLOREMQUE LAUDANTIUM, TOTAM REM APERIAM, EAQUE IPSA QUAE AB ILLO INVENTORE VERITATIS ET QUASI ARCHITECTO BEATAE VITAE DICTA SUNT EXPLICABO. NEMO ENIM IPSAM VOLUPTATEM QUIA VOLUPTAS SIT ASPERNATUR AUT ODIT AUT FUGIT, SED QUIA CONSEQUUNTUR MAGNI DOLORES EOS QUI RATIONE VOLUPTATEM SEQUI NESCIUNT. NEQUE PORRO QUISQUAM EST, QUI DOLOREM IPSUM QUIA DOLOR SIT AMET, CONSECTETUR, ADIPISCI VELIT, SED QUIA NON NUMQUAM EIUS MODI TEMPORA INCIDUNT UT LABORE ET DOLORE MAGNAM ALIQUAM QUAERAT VOLUPTATEM. UT ENIM AD MINIMA VENIAM, QUIS NOSTRUM EXERCITATIONEM ULLAM CORPORIS SUSCIPIT LABORIOSAM, NISI UT ALIQUID EX EA COMMODI CONSEQUATUR? QUIS AUTEM VEL EUM IURE REPREHENDERIT QUI IN EA VOLUPTATE VELIT ESSE QUAM NIHIL MOLESTIAE CONSEQUATUR, VEL ILLUM QUI DOLOREM EUM FUGIAT QUO VOLUPTAS NULLA PARIATUR?",
"_id":"FOO"
}To prove its all real, here is an object dump of the above JSON after it was parsed...
o=<OBJECT REFERENCE>[20@Cogs.Tests.Json.JsonClassTest]
+----------------- general information ---------------
| oref value: 20
| class name: Cogs.Tests.Json.JsonClassTest
| reference count: 2
+----------------- attribute values ------------------
| TestAllAsciiChars = " !""#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~"_$c(127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159)_" ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"
| TestBooleanNo = 0
| TestBooleanYes = 1
| TestDate = 64434
| TestEscapes = "\""\""/"""_$c(8,9,10,12,13)
| TestIgnoreFlag = "WARNING, THIS IS PRIVATE AND SHOULD NOT BE SERIALISED INTO JSON!"
| TestInteger = 42
| TestLongNumber = 49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
| TestLongString = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?"
| TestNotBoolean1 = "true"
| TestNotBoolean2 = "false"
| TestNullBoolean = ""
| TestNullDate = ""
| TestNullInteger = ""
| TestNullString = $c(0)
| TestNullTimestamp = ""
| TestRawJson = "{""TestAllAsciiChars"":"" !\""#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]"_$c(142,143,144,145,146,147,148,149,150,151,152,153,154,157,158,159)_" ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"",""TestArrayOfBoolean"":{""ALPHA"":true,""CHARLIE"":false,""DELTA"":null,""ECHO"":false,""FOXTROT"":true},""TestArrayOfDate"":{""Happy-Horolog :)"":""1840-12-31"",""doB"":""2126-12-06"",""nully"":null},""TestArrayOfInteger"":{""A THOUSAND!"":1000,""NINETY NINE"":99,""ONE"":1,""THREE"":3,""TWO"":2,""WORLD DEBT $"":59546526326827,""WORLD WORTH $"":241000000000000,""nully"":0},""TestArrayOfObject"":{},""TestArrayOfString"":{""City"":""London W1"",""Country"":""UK"",""Detective"":""Sherlock Holmes"",""District"":""Marylebone"",""FOOBAR"":""{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}"",""Street"":""221B Baker Street"",""b4d,Th1ng5!"":""\\\""\\\""/\""\b\t\r\f\n"",""not-nully"":"""",""nully"":null},""TestBooleanNo"":false,""TestBooleanYes"":true,""TestDate"":""2017-05-31"",""TestEmptyArrayOfBoolean"":{},""TestEmptyArrayOfDate"":{},""TestEmptyArrayOfInteger"":{},""TestEmptyArrayOfString"":{},""TestEmptyListOfBoolean"":[],""TestEmptyListOfOfDate"":[],""TestEmptyListOfOfInteger"":[],""TestEmptyListOfString"":[],""TestEscapes"":""\\\""\\\""/\""\b\t\r\f\n"",""TestInteger"":42,""TestListOfBoolean"":[true,false,false,null,true,false,true,true,false],""TestListOfObject"":[],""TestListOfOfDate"":[""1840-12-31"",""1841-01-10"",""1841-04-10"",""1843-09-27"",null,""1868-05-18"",""2114-10-16"",""4578-11-27""],""TestListOfOfInteger"":[0,43564356546456,0.345,0.2,0,-464356.75675,76567,3.141592653589793238,7653475667,-65753,45676.56,-6807.383887076754324],""TestListOfString"":[""Sherlock Holmes"",""221B Baker Street"",""Marylebone"",""London W1"",""UK"",null,""{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}"",""\\\""\\\""/\""\b\t\r\f\n"",""0"",""1"",""345345345345345345400""],""TestLongNumber"":49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,""TestLongString"":""Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?"",""TestNotBoolean1"":""true"",""TestNotBoolean2"":""false"",""TestNullBoolean"":null,""TestNullDate"":null,""TestNullInteger"":0,""TestNullString"":null,""TestNullTimestamp"":null,""TestRawJson"":{""data"":{""TestAllAsciiChars"":"" !\""#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUV"_$c(142,143,144,145,146,147,148,149,150,151,152,153,154,157,158,159)_" ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"",""TestArrayOfBoolean"":{""ALPHA"":true,""CHARLIE"":false,""DELTA"":null,""ECHO"":false,""FOXTROT"":true},""TestArrayOfDate"":{""Happy-Horolog :)"":""1840-12-31"",""doB"":""2125-11-27"",""nully"":null},""TestArrayOfInteger"":{""A THOUSAND!"":1000,""NINETY NINE"":99,""ONE"":1,""THREE"":3,""TWO"":2,""WORLD DEBT $"":59546526326827,""WORLD WORTH $"":241000000000000,""nully"":null},""TestArrayOfObject"":{},""TestArrayOfString"":{""City"":""London W1"",""Country"":""UK"",""Detective"":""Sherlock Holmes"",""District"":""Marylebone"",""FOOBAR"":""{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}"",""Street"":""221B Baker Street"",""b4d,Th1ng5!"":""\\\""\\\""/\""\b\t\r\f\n"",""not-nully"":"""",""nully"":null},""TestBooleanNo"":false,""TestBooleanYes"":true,""TestDate"":""2016-05-22"",""TestEmptyArrayOfBoolean"":{},""TestEmptyArrayOfDate"":{},""TestEmptyArrayOfInteger"":{},""TestEmptyArrayOfString"":{},""TestEmptyListOfBoolean"":[],""TestEmptyListOfOfDate"":[],""TestEmptyListOfOfInteger"":[],""TestEmptyListOfString"":[],""TestEscapes"":""\\\""\\\""/\""\b\t\r\f\n"",""TestInteger"":42,""TestListOfBoolean"":[true,false,false,null,true,false,true,true,false],""TestListOfObject"":[],""TestListOfOfDate"":[""1840-12-31"",""1841-01-10"",""1841-04-10"",""1843-09-27"",null,""1868-05-18"",""2114-10-16"",""4578-11-27""],""TestListOfOfInteger"":[""0"",43564356546456,0.345,0.2,null,-464356.75675,76567,3.141592653589793238,7653475667,-65753,45676.56,-6807.383887076754324],""TestListOfString"":[""Sherlock Holmes"",""221B Baker Street"",""Marylebone"",""London W1"",""UK"",null,""{[{\\}[/][{[FOO,BAR]}]{\\}[/]]}"",""\\\""\\\""/\""\b\t\r\f\n"",""0"",""1"",""345345345345345345400""],""TestLongNumber"":49947976805055875840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,""TestLongString"":""Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?"",""TestNotBoolean1"":""true"",""TestNotBoolean2"":""false"",""TestNullBoolean"":null,""TestNullDate"":null,""TestNullInteger"":null,""TestNullString"":null,""TestNullTimestamp"":null,""TestRawJson"":"""",""TestSingleObject"":"""",""TestString"":""{[{\\}[/][{[\""FOO\"",\""\""\""\""\""\""\""\""\""\""\""\""\""\""\""\""\"",\""BAR\""]}]{\\}[/]]}"",""TestStringNotNull"":"""",""TestStringOfJSON"":""{\""menu\"": { \""id\"": \""file\"", \""value\"": \""File\"", \""popup\"": { \""menuitem\"": [ {\""value\"": \""New\"", \""onclick\"": \""CreateNewdoc()\""}, {\""value\"": \""Open\"", \""onclick\"": \""Opendoc()\""}, {\""value\"": \""Close\"", \""onclick\"": \""Closedoc()\""} ] }}}"",""TestTimestamp"":""1966-01-27T23:12:02"",""TestTimestampShort"":""1966-01-27T00:00:00"",""_id"":""FOO""}},""TestSingleObject"":"""",""TestString"":""{[{\\}[/][{[\""FOO\"",\""\""\""\""\""\""\""\""\""\""\""\""\""\""\""\""\"",\""BAR\""]}]{\\}[/]]}"",""TestStringNotNull"":"""",""TestStringOfJSON"":""{\""menu\"": { \""id\"": \""file\"", \""value\"": \""File\"", \""popup\"": { \""menuitem\"": [ {\""value\"": \""New\"", \""onclick\"": \""CreateNewdoc()\""}, {\""value\"": \""Open\"", \""onclick\"": \""Opendoc()\""}, {\""value\"": \""Close\"", \""onclick\"": \""Closedoc()\""} ] }}}"",""TestTime"":""01:14:04"",""TestTimestamp"":""1966-01-27T23:12:02"",""TestTimestampShort"":""1966-01-27T00:00:00"",""TestZero"":0,""ZTestJsonMethod"":""LOREM IPSUM DOLOR SIT AMET, CONSECTETUR ADIPISCING ELIT, SED DO EIUSMOD TEMPOR INCIDIDUNT UT LABORE ET DOLORE MAGNA ALIQUA. UT ENIM AD MINIM VENIAM, QUIS NOSTRUD EXERCITATION ULLAMCO LABORIS NISI UT ALIQUIP EX EA COMMODO CONSEQUAT. DUIS AUTE IRURE DOLOR IN REPREHENDERIT IN VOLUPTATE VELIT ESSE CILLUM DOLORE EU FUGIAT NULLA PARIATUR. EXCEPTEUR SINT OCCAECAT CUPIDATAT NON PROIDENT, SUNT IN CULPA QUI OFFICIA DESERUNT MOLLIT ANIM ID EST LABORUM. SED UT PERSPICIATIS UNDE OMNIS ISTE NATUS ERROR SIT VOLUPTATEM ACCUSANTIUM DOLOREMQUE LAUDANTIUM, TOTAM REM APERIAM, EAQUE IPSA QUAE AB ILLO INVENTORE VERITATIS ET QUASI ARCHITECTO BEATAE VITAE DICTA SUNT EXPLICABO. NEMO ENIM IPSAM VOLUPTATEM QUIA VOLUPTAS SIT ASPERNATUR AUT ODIT AUT FUGIT, SED QUIA CONSEQUUNTUR MAGNI DOLORES EOS QUI RATIONE VOLUPTATEM SEQUI NESCIUNT. NEQUE PORRO QUISQUAM EST, QUI DOLOREM IPSUM QUIA DOLOR SIT AMET, CONSECTETUR, ADIPISCI VELIT, SED QUIA NON NUMQUAM EIUS MODI TEMPORA INCIDUNT UT LABORE ET DOLORE MAGNAM ALIQUAM QUAERAT VOLUPTATEM. UT ENIM AD MINIMA VENIAM, QUIS NOSTRUM EXERCITATIONEM ULLAM CORPORIS SUSCIPIT LABORIOSAM, NISI UT ALIQUID EX EA COMMODI CONSEQUATUR? QUIS AUTEM VEL EUM IURE REPREHENDERIT QUI IN EA VOLUPTATE VELIT ESSE QUAM NIHIL MOLESTIAE CONSEQUATUR, VEL ILLUM QUI DOLOREM EUM FUGIAT QUO VOLUPTAS NULLA PARIATUR?"",""_id"":""FOO""}"
| TestString = "{[{\}[/][{[""FOO"","""""""""""""""""""""""""""""""""",""BAR""]}]{\}[/]]}"
| TestStringNotNull = ""
| TestStringOfJSON = "{""menu"": { ""id"": ""file"", ""value"": ""File"", ""popup"": { ""menuitem"": [ {""value"": ""New"", ""onclick"": ""CreateNewdoc()""}, {""value"": ""Open"", ""onclick"": ""Opendoc()""}, {""value"": ""Close"", ""onclick"": ""Closedoc()""} ] }}}"
| TestTime = 4444
| TestTimestamp = "1966-01-27 23:12:02"
| TestTimestampShort = "1966-01-27 00:00:00"
| TestZero = ""
| ZTestJsonMethod = ""
| ZTestJsonName = "FOO"
+----------------- swizzled references ---------------
|i%TestArrayOfBoolean = ""
|i%TestArrayOfBoolean("ALPHA") = 1
|i%TestArrayOfBoolean("CHARLIE") = 0
|i%TestArrayOfBoolean("DELTA") = ""
|i%TestArrayOfBoolean("ECHO") = 0
|i%TestArrayOfBoolean("FOXTROT") = 1
|r%TestArrayOfBoolean = "25@%Collection.ArrayOfDT" <Set>
| i%TestArrayOfDate = ""
|i%TestArrayOfDate("Happy-Horolog :)") = 0
|i%TestArrayOfDate("doB") = 104434
|i%TestArrayOfDate("nully") = ""
| r%TestArrayOfDate = "27@%Collection.ArrayOfDT" <Set>
|i%TestArrayOfInteger = ""
|i%TestArrayOfInteger("A THOUSAND!") = 1000
|i%TestArrayOfInteger("NINETY NINE") = 99
|i%TestArrayOfInteger("ONE") = 1
|i%TestArrayOfInteger("THREE") = 3
|i%TestArrayOfInteger("TWO") = 2
|i%TestArrayOfInteger("WORLD DEBT $") = 59546526326827
|i%TestArrayOfInteger("WORLD WORTH $") = 241000000000000
|i%TestArrayOfInteger("nully") = ""
|r%TestArrayOfInteger = "28@%Collection.ArrayOfDT" <Set>
|i%TestArrayOfObject = "" <Set>
|r%TestArrayOfObject = "1@%Collection.ArrayOfObj" <Set>
|i%TestArrayOfString = ""
|i%TestArrayOfString("City") = "London W1"
|i%TestArrayOfString("Country") = "UK"
|i%TestArrayOfString("Detective") = "Sherlock Holmes"
|i%TestArrayOfString("District") = "Marylebone"
|i%TestArrayOfString("FOOBAR") = "{[{\}[/][{[FOO,BAR]}]{\}[/]]}"
|i%TestArrayOfString("Street") = "221B Baker Street"
|i%TestArrayOfString("b4d,Th1ng5!") = "\""\""/"""_$c(8,9,10,12,13)
|i%TestArrayOfString("not-nully") = ""
|i%TestArrayOfString("nully") = $c(0)
|r%TestArrayOfString = "26@%Collection.ArrayOfDT" <Set>
|i%TestEmptyArrayOfBoolean = "" <Set>
|r%TestEmptyArrayOfBoolean = "2@%Collection.ArrayOfDT" <Set>
|i%TestEmptyArrayOfDate = "" <Set>
|r%TestEmptyArrayOfDate = "3@%Collection.ArrayOfDT" <Set>
|i%TestEmptyArrayOfInteger = "" <Set>
|r%TestEmptyArrayOfInteger = "4@%Collection.ArrayOfDT" <Set>
|i%TestEmptyArrayOfString = "" <Set>
|r%TestEmptyArrayOfString = "5@%Collection.ArrayOfDT" <Set>
|i%TestEmptyListOfBoolean = "" <Set>
|r%TestEmptyListOfBoolean = "6@%Collection.ListOfDT" <Set>
|i%TestEmptyListOfOfDate = "" <Set>
|r%TestEmptyListOfOfDate = "7@%Collection.ListOfDT" <Set>
|i%TestEmptyListOfOfInteger = "" <Set>
|r%TestEmptyListOfOfInteger = "8@%Collection.ListOfDT" <Set>
|i%TestEmptyListOfString = "" <Set>
|r%TestEmptyListOfString = "9@%Collection.ListOfDT" <Set>
|i%TestListOfBoolean = ""
|i%TestListOfBoolean(1) = 1
|i%TestListOfBoolean(2) = 0
|i%TestListOfBoolean(3) = 0
|i%TestListOfBoolean(4) = ""
|i%TestListOfBoolean(5) = 1
|i%TestListOfBoolean(6) = 0
|i%TestListOfBoolean(7) = 1
|i%TestListOfBoolean(8) = 1
|i%TestListOfBoolean(9) = 0
|r%TestListOfBoolean = "22@%Collection.ListOfDT" <Set>
| i%TestListOfObject = "" <Set>
| r%TestListOfObject = "10@%Collection.ListOfObj" <Set>
| i%TestListOfOfDate = ""
|i%TestListOfOfDate(1) = 0
|i%TestListOfOfDate(2) = 10
|i%TestListOfOfDate(3) = 100
|i%TestListOfOfDate(4) = 1000
|i%TestListOfOfDate(5) = ""
|i%TestListOfOfDate(6) = 10000
|i%TestListOfOfDate(7) = 100000
|i%TestListOfOfDate(8) = 1000000
| r%TestListOfOfDate = "23@%Collection.ListOfDT" <Set>
|i%TestListOfOfInteger = ""
|i%TestListOfOfInteger(1) = 0
|i%TestListOfOfInteger(2) = 43564356546456
|i%TestListOfOfInteger(3) = .345
|i%TestListOfOfInteger(4) = .2
|i%TestListOfOfInteger(5) = ""
|i%TestListOfOfInteger(6) = -464356.75675
|i%TestListOfOfInteger(7) = 76567
|i%TestListOfOfInteger(8) = 3.141592653589793238
|i%TestListOfOfInteger(9) = 7653475667
|i%TestListOfOfInteger(10) = -65753
|i%TestListOfOfInteger(11) = 45676.56
|i%TestListOfOfInteger(12) = -6807.383887076754324
|r%TestListOfOfInteger = "24@%Collection.ListOfDT" <Set>
| i%TestListOfString = ""
|i%TestListOfString(1) = "Sherlock Holmes"
|i%TestListOfString(2) = "221B Baker Street"
|i%TestListOfString(3) = "Marylebone"
|i%TestListOfString(4) = "London W1"
|i%TestListOfString(5) = "UK"
|i%TestListOfString(6) = $c(0)
|i%TestListOfString(7) = "{[{\}[/][{[FOO,BAR]}]{\}[/]]}"
|i%TestListOfString(8) = "\""\""/"""_$c(8,9,10,12,13)
|i%TestListOfString(9) = 0
|i%TestListOfString(10) = 1
|i%TestListOfString(11) = 345345345345345345400
| r%TestListOfString = "21@%Collection.ListOfDT" <Set>
| i%TestSingleObject = ""
| r%TestSingleObject = ""
+-----------------------------------------------------Btw, the main part of this post was written in the markdown editor and cut and paste directly into this post with syntax highlighting for COS.
Sean.
Outstanding!
I'll keep a close eye on that thread.
Since you're using something close to decorators/annotations, does that means you made a COS syntax parser, lexer and AST?
I'm also very interested about how fast and what strategies you used for parsing JSONs.
Here is a techical background:
So far, when realising performance tests I had to opt-out from using the default %Stream classes's MoveTo method, since it always rewinds the position and moves the pointer , this caused me great performance headaches. So I had to implement some sort of algorithm that lazy loads and calculates the position.
I'm wondering if you had to implement something focused on older Caché versions and faced such issues, because that was my case. I had to implement a JSON parser for the 2010 version, due to our client and partner restrictions.
Hi Rubens,
I designed the solution around the real life use cases that I hit in my mainstream work.
In most instances I am handling JSON to and from a browser and I have never had a use case where the JSON is over Cachés long string support of 3,641,144 characters.
Thats with the exception of wanting to post a file with JSON. In that instance I have some boiler plate code that sends them as multiparts and joins them back together after the main JSON parse.
With those decisions made it was just a matter of writing very efficient COS code that processed long strings. A couple of years ago the serialiser and deserialiser classes stacked up pretty big. In this latest version they are an uber efficient 90 and 100 lines of code each.
There is no AST magic going on, just projection compilation with inspection of the dictionary. A small lib to abstract the annotations and various code generator tricks to bake in type handlers and deligators.
Where data might go over 3,641,144 characters is parsing data backwards and forwards with Node.JS or another Caché server. In this instance the data is almost always going to be an array of results or an array of objects. For the later there is a large array helper class I am working on that will split out individual objects from a stream and then handle them as long strings. This will be part of the Node package.
In the few fringe cases where someone might be generating objects larger than 3,641,144 characters then it wouldn't be to hard to have stream varients. I used to have these, but dropped them because they were never used. But I would keep the string handler variants as the primary implementations as they prove very quick.
As for older Caché instances, I have had to support JSON as long as 8 years ago and still see the need for backwards compatibility.
Sean.
90 and 100 lines for serializers is quite the achievement.
Oh, I see, so that's how you introduced decorators.
Now about JSON parsing...
In my case I had to provide out-of-the-box support for streams and keep the parsing process the faster as I could, since we intended to use it for parsing files that weighted around 5~8 MBs to populate the database. CSV or XML wasn't an option at that time.
I can provide the code for that.
About NodeJS ORM:
Also, how are you connecting NodeJS to Caché? Is it using the native driver provided by InterSystems? If so, are you dealing with globals directly or did you implemented an adapter for NoSQL?
I ended up writing my own solution in the end.
It's a TCP wire based solution that uses the JSON-RPC messages as the main protocol.
Node starts up a concurrent TCP listener and then Caché jobs off as many client connections as required.
It surprisingly simple on the Node side, minimal glue to bind HTTP requests to TCP messages with zero blocking.
I did quit a lot of testing on it at the time I wrote it and found that I could get twice as many RPC messages into Cache via Node than I could via CSP. My guess is that the RPC route does not have to deal with all the HTTP protocols.
I then wrapped the same event emitter used for the HTTP requests with a small promise caller and was able to do some testing of proxy objects inside Node itself. It's a little bit experimental on the Node side, but I am able to run the 30,000 browser unit tests (lots of automated ones in there) over the ORM library and it just works.
Not sure I would want to put it into production until its been kicked around some more.
Interesting...
Can you provide some example of usage for that node package?
Hi Sean.
This looks interesting.
Why do you prefer annotations style instead of standard Caché attribute style for properties?
I mean
/// @JSONNAME=BirthDate Property DateOfBirth As %Date;Instead of
Property DateOfBirth As %Date(JSONNAME = "BirthDate");Thank you,
Alexander.
Hi Alexander,
Unless I am missing a cool trick, you can't do this directly...
Property DateOfBirth As %Date(JSONNAME = "BirthDate");
you would need to extend %Date with your own class type and add the JSONNAME parameter to it, which means you end up with...
Property DateOfBirth As Cogs.Lib.Types.Date(JSONNAME = "BirthDate");
Which for me feels much more cumbersome, not to mention that developers are forced to change all of their existing code as well as amend any existing overriden data types that they use.
Unless I am missing another trick, I'm pretty sure you can't add these attributes to complex types which if I am right is a show stopper anyway.
Annotations are just much easier to work with, I need them for methods as well so it just seems more in keeping to do it all this way.
Sean.
You can add extra parameters for all properties of particular class using PropertyClass keyword
http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=...
This does not apply to methods though...
It does(EDIT: nope, it doesn't, because it's easier). I use it for publishing (or making RPC-aware methods).And it doesn't even requires PropertyClass, instead of properties.
Yeah, I also had to do something along these lines.
Though I can't publish the source code for now, since it does contains business related authentication code.
OK, excellent thanks for that. Seem to remember hitting a brick wall trying to get this to work many moons ago.
Here's how we did.
It works for 2010, in that case I expose the controller methods and using &p1=id ... &pN=idN to retrieve a JSON collection of
SamplePerson serialized instances.
Notice PUBLIC, methods that have this parameter are available to be called over HTTP.
{
set persons = ##class(%ListOfObjects).%New()
set sumEq = ""
set sum = 0
for i=1:1:args {
do persons.Insert(args(i))
}
quit persons
}
I'll check if I can remove the proprietary code and make it available as well.
Oh. OK. (PUBLIC=1) in this case is related to %ListOfObjects return class, not the method itself.
Although, it's the question of how developer interpret this parameter.
Yeah, this approach forces the user to specify the return type.
Excellent, just tried it and the value is accessible via ReturnTypeParams in the %Dictionary.CompiledMethod table.
RESTForms uses this approach.
See the example.
Thanks Rubens and Alexander, I've not even released the code and getting good ideas to improve things - open source at its best.
Given that return types can also be applied to methods, I am now weighing up native vs annotations.
Any preferences?
I do suppose it's easier to extract parameters from ReturnTypeParams than parsing then through property documentations. Especially if there's a case with multiple parameters. So I think that's more a technical debt situation than an end-user preference.
At the moment I have a serializer (<glvn> to json string) and a deserializer (json string to <glvn>) of 40-60 lines of pure
MCOS code. Should run on any Caché version, was tested on 2012.2 and higher. Some proof that they really exist:USER>d $system.CPU.Dump() -- CPU Info for node maslov -------------------------------------------------- Architecture: x86 Model: Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz < ... > USER>d j2gSmall^zmawr JSON2G^Wmgr Cache for Windows (x86-64) 2017.1 total time = 1.05795 avg time = .0000105795 G2JSON^Wmgr Cache for Windows (x86-64) 2017.1 total time = 1.898275 avg time = .00001898275 sJson <=> arr()? yes ; the result of conversion reversibility check USER>zw sJson sJson="{""BirthDate"":""1970-03-25"",""FirstName"":""Sean"",""Hobbies"":[""Photography"",""Walking"",""Football""],""LastName"":""Connelly""}" USER>zw arr arr("BirthDate")="1970-03-25" arr("FirstName")="Sean" arr("Hobbies",0)="Photography" arr("Hobbies",1)="Walking" arr("Hobbies",2)="Football" arr("LastName")="Connelly"This code is not my own development (while I contributed a bit), so if anybody wants it to be published I should redirect this request to main contributor(s).
BTW: Sean, was this JSON string
TestRawJson = "{""TestAllAsciiChars"":"" !\""#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]"_$c(142,143,144,145,146,147,148,149,150,151,152,153,154,157,158,159)_"...from your sample wrong escaped deliberately?
Hi Alexy,
You've fished out a property that is of type Cogs.Lib.Types.Json,
In its property state the JSON is stored as a pure string, hence seeing the odd escaping.
When its serialized back out to JSON it will be correctly escaped, which you can see in the JSON dump I posted before it.
This provides the best of both worlds, schema driven properties that can have one or more non schema properties for generic data storage.
btw, Cogs includes JSON classes for serialising and de-serialising to and from arrays and globals as well, interestingly they are only 50 lines of code each, so will be interesting to compare them.
Sean.
It's portuguese though. But I think you can figure the essentials.
There's a few things to notice (and some to fix).
1 - The biggest file is MG.json, it's a JSON containing all coordinates to render the biggest state from here. You can see it takes about 6.5 minutes to parse all data. But it's a special case, not something common to be parsed. Btw, I said 5~8 MB, but eh... it's about 3 MB (3.810.312 bytes), my mistake.
2 - blns.json.txt is a nasty sure-fire JSON file. I took it from this repo and trying to figure how to solve a parsing issue. If you want something to risk breaking your parser, really, try this file.
3 -Some chars below might not be shown correctly, that is due to unicode support. That's because of assertions related to unescaping paired unicode sequences like:
tests begins ... UnitTest.JQX.JSON.Parser begins ... TestArrayMiscParsing() begins ... AssertEquals:Array precisa conter 6 valores (passed) AssertEquals:Posição 1 do array deve ser um objeto (passed) AssertEquals:Posição 1 do array deve ser um proxy (passed) AssertEquals:Proxy deve conter propriedade 'foo' definido com valor 'bar' (passed) AssertEquals:Proxy deve conter propriedade 'foo' definida como outro proxy (passed) AssertEquals:Posição 2 do array deve um %ListOfDataTypes (passed) AssertEquals:Deve conter 5 valores (passed) AssertEquals:Posição 1 deve ser ter valor 1 (passed) AssertEquals:Posição 2 deve ser ter valor 2 (passed) AssertEquals:Posição 3 deve ser ter valor 3 (passed) AssertEquals:Posição 4 deve ser um %ListOfDataTypes (passed) AssertEquals:O array dessa posição deve ter um item (passed) AssertEquals:Com o valor 15 (passed) AssertEquals:Posição 5 deve ser um proxy (passed) AssertEquals:O proxy deve ter uma propriedade 'well' com valor 'done' (passed) AssertEquals:De volta ao array principal, a posição 3 deve ser null (passed) AssertEquals:De volta ao array principal, a posição 4 deve ser 'null' equivalendo null (passed) AssertEquals:De volta ao array principal, a posição 5 deve ser true (passed) AssertEquals:De volta ao array principal, a posição 6 deve ser 'true' equivalendo true (passed) LogMessage:Duration of execution: .269543 sec. TestArrayMiscParsing passed TestArrayNumberParsing() begins ... AssertEquals:Array precisa conter 3 valores (passed) AssertEquals:Posição 1 do array deve ser 1 (passed) AssertEquals:Posição 2 do array deve ser 2 (passed) AssertEquals:Posição 3 do array deve ser 2.50 (passed) LogMessage:Duration of execution: .045573 sec. TestArrayNumberParsing passed TestArrayReservedParsing() begins ... AssertEquals:Array precisa conter 3 valores (passed) AssertEquals:Posição 1 do array deve ser true (passed) AssertEquals:Posição 2 do array deve ser false (passed) AssertEquals:Posição 3 do array deve ter null (passed) LogMessage:Duration of execution: .047348 sec. TestArrayReservedParsing passed TestArrayStringParsing() begins ... AssertEquals:Array precisa conter 3 valores (passed) AssertEquals:Posição 1 do array deve ter 'quite' (passed) AssertEquals:Posição 2 do array deve ter 'some' (passed) AssertEquals:Posição 3 do array deve ter 'string' (passed) LogMessage:Duration of execution: .052526 sec. TestArrayStringParsing passed TestEscapedStringParsing() begins ... AssertEquals:Deve retornar um sorriso �� (passed) AssertEquals:'a' deve ter valor 1 (passed) AssertEquals:'c' deve ter conter 3 itens (passed) AssertEquals:A primeira posição deve conter o sorriso de novo �� (passed) AssertEquals:Deve ter a seguinte mensagem: Isso é um "teste" �� (passed) AssertEquals:Deve conter um objeto vazio (passed) LogMessage:Duration of execution: .104971 sec. TestEscapedStringParsing passed TestMultilineParsing() begins ... LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo blns.json.txt <!> *** LogMessage:******************************************************* LogMessage: ERRO #5001: Syntax Error: Expected value separator or array ending, but found ASCII code 92 at offset 207 LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo example.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: .07764 LogMessage:Início do parsing: 05/31/2017 13:43:38 LogMessage:Fim do parsing: 05/31/2017 13:43:38 AssertEquals:Deve retornar um objeto a partir do arquivo example.json (passed) AssertEquals:id deve ter valor igual a string 0001 (passed) AssertEquals:type deve ter valor igual a string donut (passed) AssertEquals:name deve ter valor igual a string Cake (passed) AssertEquals:ppu deve ter valor numerico igual .55 (passed) AssertEquals:A propriedade batter deve ter 4 itens (passed) AssertEquals:id de batter deve ter valor igual 1001 (passed) AssertEquals:type de batter deve ter valor igual Regular (passed) AssertEquals:id de batter deve ter valor igual 1002 (passed) AssertEquals:type de batter deve ter valor igual Chocolate (passed) AssertEquals:id de batter deve ter valor igual 1003 (passed) AssertEquals:type de batter deve ter valor igual Blueberry (passed) AssertEquals:id de batter deve ter valor igual 1004 (passed) AssertEquals:type de batter deve ter valor igual Devil's Food (passed) AssertEquals:id de topping deve ter valor igual 5001 (passed) AssertEquals:type de topping deve ter valor igual None (passed) AssertEquals:id de topping deve ter valor igual 5002 (passed) AssertEquals:type de topping deve ter valor igual Glazed (passed) AssertEquals:id de topping deve ter valor igual 5005 (passed) AssertEquals:type de topping deve ter valor igual Sugar (passed) AssertEquals:id de topping deve ter valor igual 5007 (passed) AssertEquals:type de topping deve ter valor igual Powdered Sugar (passed) AssertEquals:id de topping deve ter valor igual 5006 (passed) AssertEquals:type de topping deve ter valor igual Chocolate with Sprinkles (passed) AssertEquals:id de topping deve ter valor igual 5003 (passed) AssertEquals:type de topping deve ter valor igual Chocolate (passed) AssertEquals:id de topping deve ter valor igual 5004 (passed) AssertEquals:type de topping deve ter valor igual Maple (passed) AssertEquals:id deve ter valor igual a string 0002 (passed) AssertEquals:type deve ter valor igual a string donut (passed) AssertEquals:name deve ter valor igual a string Raised (passed) AssertEquals:ppu deve ter valor numerico igual .55 (passed) AssertEquals:A propriedade batter deve ter 1 itens (passed) AssertEquals:id de batter deve ter valor igual 1001 (passed) AssertEquals:type de batter deve ter valor igual Regular (passed) AssertEquals:id de topping deve ter valor igual 5001 (passed) AssertEquals:type de topping deve ter valor igual None (passed) AssertEquals:id de topping deve ter valor igual 5002 (passed) AssertEquals:type de topping deve ter valor igual Glazed (passed) AssertEquals:id de topping deve ter valor igual 5005 (passed) AssertEquals:type de topping deve ter valor igual Sugar (passed) AssertEquals:id de topping deve ter valor igual 5003 (passed) AssertEquals:type de topping deve ter valor igual Chocolate (passed) AssertEquals:id de topping deve ter valor igual 5004 (passed) AssertEquals:type de topping deve ter valor igual Maple (passed) AssertEquals:id deve ter valor igual a string 0003 (passed) AssertEquals:type deve ter valor igual a string donut (passed) AssertEquals:name deve ter valor igual a string Old Fashioned (passed) AssertEquals:ppu deve ter valor numerico igual .55 (passed) AssertEquals:A propriedade batter deve ter 2 itens (passed) AssertEquals:id de batter deve ter valor igual 1001 (passed) AssertEquals:type de batter deve ter valor igual Regular (passed) AssertEquals:id de batter deve ter valor igual 1002 (passed) AssertEquals:type de batter deve ter valor igual Chocolate (passed) AssertEquals:id de topping deve ter valor igual 5001 (passed) AssertEquals:type de topping deve ter valor igual None (passed) AssertEquals:id de topping deve ter valor igual 5002 (passed) AssertEquals:type de topping deve ter valor igual Glazed (passed) AssertEquals:id de topping deve ter valor igual 5003 (passed) AssertEquals:type de topping deve ter valor igual Chocolate (passed) AssertEquals:id de topping deve ter valor igual 5004 (passed) AssertEquals:type de topping deve ter valor igual Maple (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo MG.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: 395.318218 LogMessage:Início do parsing: 05/31/2017 13:43:39 LogMessage:Fim do parsing: 05/31/2017 13:50:14 AssertEquals:Deve retornar um objeto a partir do arquivo MG.json (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo municipios.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: 10.035081 LogMessage:Início do parsing: 05/31/2017 13:50:15 LogMessage:Fim do parsing: 05/31/2017 13:50:25 AssertEquals:Deve retornar um objeto a partir do arquivo municipios.json (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo package.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: .057244 LogMessage:Início do parsing: 05/31/2017 13:50:25 LogMessage:Fim do parsing: 05/31/2017 13:50:25 AssertEquals:Deve retornar um objeto a partir do arquivo package.json (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo random.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: .518465 LogMessage:Início do parsing: 05/31/2017 13:50:25 LogMessage:Fim do parsing: 05/31/2017 13:50:25 AssertEquals:Deve retornar um objeto a partir do arquivo random.json (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo random_bigger.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: 1.920557 LogMessage:Início do parsing: 05/31/2017 13:50:25 LogMessage:Fim do parsing: 05/31/2017 13:50:27 AssertEquals:Deve retornar um objeto a partir do arquivo random_bigger.json (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo schema.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: .03808 LogMessage:Início do parsing: 05/31/2017 13:50:27 LogMessage:Fim do parsing: 05/31/2017 13:50:27 AssertEquals:Deve retornar um objeto a partir do arquivo schema.json (passed) AssertEquals:Deve conter uma propriedade $schema com o valor http://json-schema.org/draft-04/schema# (passed) AssertEquals:Deve conter um objeto chamado 'definitions' (passed) AssertEquals:Deve conter um objeto chamado 'address' (passed) AssertEquals:Deve conter uma propriedade 'type' em 'address' com o valor object (passed) AssertEquals:Deve conter um objeto chamado 'properties' em 'address' (passed) AssertEquals:Deve conter um objeto chamado 'street_address' em 'properties' (passed) AssertEquals:Deve conter uma propriedade 'type' em 'street_address' com o valor string (passed) AssertEquals:Deve conter um objeto chamado 'city' em 'properties' (passed) AssertEquals:Deve conter uma propriedade 'type' em 'city' com o valor string (passed) AssertEquals:Deve conter um objeto chamado 'state' em 'properties' (passed) AssertEquals:Deve conter uma propriedade 'type' em 'state' com o valor string (passed) AssertEquals:Deve conter uma coleção em 'address' chamada 'required' (passed) AssertEquals:Deve conter 3 valores na coleção mencionada (passed) AssertEquals:Posicao 1 deve ter 'street_address' (passed) AssertEquals:Posicao 2 deve ter 'city' (passed) AssertEquals:Posicao 3 deve ter 'state' (passed) LogMessage: LogMessage:******************************************************* LogMessage:*** <!> Testando arquivo youtube.json <!> *** LogMessage:******************************************************* LogMessage: LogMessage:Duração em segundos: .092909 LogMessage:Início do parsing: 05/31/2017 13:50:27 LogMessage:Fim do parsing: 05/31/2017 13:50:28 AssertEquals:Deve retornar um objeto a partir do arquivo youtube.json (passed) LogMessage:Duration of execution: 409.506008 sec. TestMultilineParsing passed TestObjectKeyArrayValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor de um array (passed) LogMessage:Duration of execution: .012107 sec. TestObjectKeyArrayValueParsing passed TestObjectKeyFalseValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor true (passed) LogMessage:Duration of execution: .012377 sec. TestObjectKeyFalseValueParsing passed TestObjectKeyNullValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor null (passed) LogMessage:Duration of execution: .015294 sec. TestObjectKeyNullValueParsing passed TestObjectKeyObjectValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor de um proxy (passed) LogMessage:Duration of execution: .01224 sec. TestObjectKeyObjectValueParsing passed TestObjectKeyStringValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor 'string' (passed) LogMessage:Duration of execution: .012922 sec. TestObjectKeyStringValueParsing passed TestObjectKeyTrueValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor true (passed) LogMessage:Duration of execution: .012159 sec. TestObjectKeyTrueValueParsing passed TestObjectMultiKeyObjectValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' como sendo um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'b' como sendo um proxy (passed) AssertEquals:'b' deve ter foo definido com valor 'bar (passed) LogMessage:Duration of execution: .024768 sec. TestObjectMultiKeyObjectValueParsing passed TestObjectMultiKeyStringValueParsing() begins ... AssertEquals:Objeto deve ser um proxy (passed) AssertEquals:Objeto precisa conter propriedade 'a' com valor 'string' (passed) AssertEquals:Objeto precisa conter propriedade 'b' com valor 'foo' (passed) LogMessage:Duration of execution: .019991 sec. TestObjectMultiKeyStringValueParsing passed UnitTest.JQX.JSON.Parser passed Skipping deleting classes tests passedAnd here's the fixtures used for this test.
https://mega.nz/#!USAlUZBS!zhmdlL-ovp0mq8ACic2BBqia_HxJWqmBwXFkiMkbmgQ
When I've seen "ORM javascript" in the feature list I remembered yet another very interesting approach which is in fact the result of Hachathon took place in Moscow two years ago: CNDO (Caché Node Data Object), project made by [@Nikita.Savchenko], Anton Gnibeda and [@Irene.Mikhaylova].
And here is the link to our project :)
Looks good Nikita, are you using this under WebTerminal?
Thanks Sean!
No. It was build over REST indeed while WebTerminal, from very first its version used WebSockets. However, the same interface can be implemented over WebSockets :) In WebTerminal, there is a simple JSON messaging, there is no need to add more tools there I think.
As requested, here are some snippets of the ORM library that works for both browser and Node.JS. This is from some of 30,000 unit tests that I built on top of the Northwind database data.
The solution starts with a Caché class that extends the Cogs.Store class, this is just a normal %Persistent class with extra methods.
Class Cogs.CoffeeTable.Tests.Northwind.Customers Extends Cogs.Store { Parameter DOMAIN = "northwind"; Property CustomerID As %String; Property CompanyName As %String; Property ContactName As %String; Property ContactTitle As %String; Property Address As %String; Property City As %String; Property Region As %String; Property PostalCode As %String; Property Country As %String; Property Phone As %String; Property Fax As %String; Index CustomerIDIndex On CustomerID [ IdKey, PrimaryKey, Unique ]; }There are then two approaches to develop in JavaScript. The first is to include a client API script that is dynamically created on the fly, this includes a promise polyfill and an HTTP request wrapper. This is a good approach for small to medium projects.
In this instance there will be a global object called northwind that will contain a set of database objects, each with a set of CRUD methods
A basic example of using find...
northwind.customers.find().then( function(data) { console.log(data) } )The second approach uses TypeScript and Browserify using a modern ES6 approach.
A code generator produces a TypeScript Customer schema class...
import {Model} from 'coffeetable/Model'; export class CustomerSchema extends Model { static _uri : string = '/northwind/customers'; static _pk : string = 'CustomerID'; static _schema = { Address : 'string', City : 'string', CompanyName : 'string', ContactName : 'string', ContactTitle : 'string', Country : 'string', Fax : 'string', Phone : 'string', PostalCode : 'string', Region : 'string', CustomerID : 'string' }; CustomerID : string; Address : string; City : string; CompanyName : string; ContactName : string; ContactTitle : string; Country : string; Fax : string; Phone : string; PostalCode : string; Region : string; }as well as a model class which can then be extended without affecting the generated class...
import {CustomerSchema} from '../schema/Customer'; export class Customer extends CustomerSchema { //extend the proxy client class here }now I can develop a large scale application around these proxy objects and benefit from schema validation, auto type conversions as well as having object auto complete inside IDE's such as WebStorm.
Create and save a new object...
import {Customer} from "./model/Customer"; var customer = new Customer(); //Each one of these properties auto completed customer.CustomerID = record[0]; customer.CompanyName = record[1]; customer.ContactName = record[2]; customer.ContactTitle = record[3]; customer.Address = record[4]; customer.City = record[5]; customer.Region = record[6]; customer.PostalCode = record[7]; customer.Country = record[8]; customer.Phone = record[9]; customer.Fax = record[10]; customer.save().then( (savedCustomer : Customer) => { console.log(customer) }).catch( err => { console.log(err) })Open it...
Customer.open('ALFKI').then( customer => { console.log(customer.CompanyName); })Search...
Customer.find({ where : "City = 'London' AND ContactTitle = 'Sales Representative'" }).then( customers => { console.log(customers); });The last example returns a managed collection of objects. In this instance the second approach includes a more sophisticated client library to work with the collection, such that you can filter and sort the local array without needing to go back to the server.
customers.sort("Country")this triggers a change event on the customers collection which would have been scoped to a view, so for instance you might have a React component that subscribes to the change and sets its state when the collection changes
Motivation
I needed to develop an application that could run on existing customer databases, Ensemble -> Caché, Mirth -> PostgreSQL as well as MongoDB. Such that the database can be swapped in and out without changing a line of client code.
I looked to adapt one of the existing ORM libraries such as Sequelize or Sails but it was easier to start out from scratch to leverage on Caché without needing to use lots of duck tape to get it working.
This new solution required a JSON-RPC interface and more JSON functionality from Caché, hence re-engineering some old JSON libs and building out the Cogs library.
Moving forward the plan is to release CoffeeTable as a seperate NPM library and Cogs will essentially be a server side adapter to it.
Probably the wrong forum to talk about GT.m, but I have a long standing internal library that was designed for this eventual abstraction and will be one of the databases that will be added to CoffeeTable down the line.
Thanks for posting about it's usage.
It's looks pretty straightforward and with a low learning curve.
You mentioned earlier about using TCP over CSP, does that means you implemented a messaging protocol over TCP layer instead of using HTTP?
Is that lib restricted for NodeJS usage or it could be webpack'ed for example?Nevermind, I just re-read the part where you mentioned about client API.How do you deal with license usage? How much does it escalates with a fair amount of users and how do you manage all of that?
Its a very simple JSON-RPC wire protocol. The JSON is stripped of formatting. Its then delimited with ASCII 13+10 which are already escaped in the JSON. Nothing more complicated than that.
> How do you deal with license usage? How much does it escalates with a fair amount of users and how do you manage all of that?
I can only refer to benchmarks at the moment, hence why the node connector is still marked as experimental.
The set up was on a single 3 year old commodity desktop machine running a stress tool, node, cache and about 10 other open applications.
The stress tool would simulate 50 users sending JSON-RPC requests over HTTP to a Node queue, a single Caché process would collect these requests over TCP, unpack the JSON, perform a couple of database operations, create a response object, serialise it and pass it all the way back.
With one single Caché process running one single licence I recorded an average of 1260 requests per second.
I see, is there some kind of security implemented or planned? Like an authentication mechanism.
Also, one single license. I'm starting to see how things work now, if I understood correctly most of the heavy-lifting is done using NodeJS.
Similar to sending a payload with multiple requests from multiple client connections, limited by a threshould, let's say... about 1000 requests. So, Caché simply resolves all of them and return them at once. NodeJS looks what's relevant for each client and forwards the filtered response.
But that is only possible if you host an intermediate server, do you use something like Express?
I don't know why you don't use the cache.node interface? It will support in excess of 100k global sets/second per connection. Is there something you believe it doesn't do that you need?
100,000 per second is a synthetic benchmark, a for loop in a terminal window will only just do 100,000 global sets a second, and this is without any data validation, data loops, referential integrity etc
you also don't mention if this is done via the API or over the network, I would only be interested in the over the network benchmarks
what I would be really interested in are real world benchmarks that track the number of http requests handled per second, so not some tight benchmark loop, but real end to end http requests from browser, federated through node, to cache.node and Caché and back again
plus I am not really interested in global access from node, I want to work with objects everywhere and gain the performance of letting optimised queries run on Caché without shuffling data back and forth unnecessarily
i know cache.node does handle objects, but it just doesn't fit my needs, I'm not a fan of the API and it is missing some functionality that i need
fundamentally there is a missmatch with the CoffeeTable framework that I have developed and the cache.node API
basically it just didn't seem like a good idea to end up using cache.node as nothing more than a message forwarder with potential overhead that I can't see, what I ended up with is a lean 142 lines of node code that is practically idling in the benchmarks that I have done so far
i also have concerns over the delays I have read about with the cache.node versions keeping up with the latest Node.JS version
the other thing is where is its open source home, I looked and couldn't find it, would have been nice to inspect the code, see how it works and fill in the gaps that the documentation does not go deep enough into
ultimately, why not have alternatives, different solutions for different needs
Perhaps someone from InterSystems should respond with respect to Sean's views on cache.node?
Rubens,
I would be interested in trying out your pre-release JSON library.
Thanks,
Skip
skip@pathview.com
Greetings Skip,
Just so you know. Unless you aren't using a version that already supports JSON, I can say that %DynamicAbstractObject's %FromJSON and %ToJSON is from far the fastest parsing/serializing implementation. It's about to 4x ~ 8x faster than my lib.
Otherwise I see no problem on open sourcing this library. But before I do that, I need to remove business related implementations.
Nice. Keep an eye on this post. Great job so far.
Hi Sean!
Is it possible to serialize global to JSON and export into a file with Cogs?
and vice-versa - import JSON to a global?
Hi Evgeny,
The current GitHub version will only serialise and deserialise to and from class based objects.
I do however have several other solutions in the unofficial version which will efficiently serialise and deserialise to and from globals. I also have a pollyfill solution for DynamicObject and DynamicArray that uses a type mixer class that would allow older versions of Cache to work with these classes now.
However, I've not used these in production, only unit tested. I am happy to release them if there is a need / someone is willing to collaborate on production level testing and debugging.
Sean, I cannot promise collaboration) But happy to test and submit issues if any )