W3 Systems W3S Incorporated


Not to be confused with the W3C (the international web standards organization.)

The W3SEA is a new Systems Engineering Architecture (with an alternate, equally valid meaning: 'Science and Engineering Architecture'.)  Written in C++. It's designed to maximize re-use of a "unit of function" through a design paradigm that plugs processors into a process mesh. The W3SEA has a small running memory footprint, and has low computational overhead. It's fast.  It also has a robust signaling system that works across process meshes and can signal to remote W3SEA instances over the Internet.  This allows the system to be distributed.

The W3SEA can interface with preexisting systems, such as Apache, Asterisk, PJSIP, and various SMTP, OMTP, and IMAP services, or it can completely replace them.

The ultimate goal is to replace them.

The W3SEA can be used in different domains.  It should do very well in web services, protocol, simulation, signal analysis, data mining, distributed systems, robotics, and artificial intelligence. 

Building Software

On Linux, the typical way to build binaries is by creating and using configure and make scripts.  These sort of work, but  I have never been on a project or looked at an open source project where I liked these scripts.  Descriptions like archaic, obscure, and cryptic come to mind. There are usually hacks involved to glue the scripts together and make them work properly. On the W3sea framework, I'm giving up on these old 'venerable' tools.

Python is actually a good language for creating build libraries that I believe will be easier to use, and will be much more transparent and flexible.  So W3 Systems has moved to a python based build system.

Example make.py file using the Rules.py library:

import sys
import os

from Rules import Rules
from Rules import Rule
print("Buiding Main...")

src_dir = os.getcwd();
tgt_dir = src_dir.replace("w3Sea","w3SeaBuilt")

dir_action = "mkdir -p {TGT} && touch {TGT}" 
dirs = [ "common","processors.v2", "testlib", "cgi", "http", "database", "model", "web_ux", "apps/humans" ]

db_inc = " -I/usr/include/mysql/"
db_link = " -L/usr/lib/x86_64-linux-gnu -lpthread -lz -lm -lrt -ldl -lmysqlclient"
fs_link = " -lstdc++fs "
crypto_link = " -lcrypto"
incpaths = ""
for direc in dirs:
  incpaths += " -I"+src_dir+"/"+direc

cc = "g++ -g -std=c++14 "
compile_action = cc + incpaths + db_inc + " -c {SRC} -o {TGT}"

link_test_action = cc + incpaths + db_inc + """ {SRC} {INTERMEDIATES} -o {TGT} """ + db_link + fs_link + crypto_link;

clean_action = "rm -f {TGT}"

submake_action = "python3 {SRC} "

builder = Rules()

# build rules for the target directory structure
for direc in dirs:
  builder.add( "structure", Rule(src_dir, tgt_dir, direc, direc, dir_action, True) )

# build rules for the target object files
for direc in dirs:
  builder.add( "objects", Rule(src_dir, tgt_dir, direc+"/*.cpp", direc+"/*.o", compile_action, True) )

# build rule for test app
builder.add("link_test", Rule(src_dir, tgt_dir, "Test.cpp", "runtest", link_test_action, True, None,
 [ "common/data.o", "common/strings.o", "common/time_date.o", "common/template_parser.o", "processors.v2/processor.o", 
 "processors.v2/routelet.o", "processors.v2/siglet.o", "processors.v2/router.o", "database/db_connection_mysql.o", "database/db_cursor.o", 
 "database/db_cursor_mysql.o",  "database/db_field.o", "database/db_statement.o", "database/db_statement_mysql.o",  "database/db_factory.o", 
 "model/session.o",  "model/persistence.o", "processors.v2/router_routelet.o",  "common/base64_encoder.o",  "common/crypto_hash.o", 
 "database/db_config.o",  "common/json_serializer.o",  "model/session_persists.o",  "common/lockbox.o",  "common/types.o" ]) )

#build rules for cleaning
for direc in dirs:
  builder.add( "clean", Rule(src_dir, tgt_dir, None, direc+"/*.o", clean_action) )

builder.add( "clean", Rule(src_dir, tgt_dir, None, "runtest", clean_action) )
builder.add( "cleanhumans", Rule(src_dir, tgt_dir, "apps/humans/make.py", None, submake_action+"clean") );
builder.add( "cleancgi", Rule(src_dir, tgt_dir, "cgi/make.py", None, submake_action+"clean") );

builder.add( "submake", Rule(src_dir, tgt_dir, "apps/humans/make.py", None, submake_action,True) );
builder.add( "submake", Rule(src_dir, tgt_dir, "cgi/make.py", None, submake_action,True) );

builder.add( "deploy", Rule(src_dir, tgt_dir, "apps/humans/make.py", None, submake_action+"deploy") );
builder.add( "deploy", Rule(src_dir, tgt_dir, "cgi/make.py", None, submake_action+"deploy") );
builder.add( "deploy", Rule(src_dir, tgt_dir, "web_ux/make.py", None, submake_action+"deploy") );

# Finds changes in include files: Make sure the group braces in the last r.e. pattern encompasses the file only!
builder.find_dependencies(".*.cpp","#include "(.*.h)"",)

#now build anything that isn't up to date.

Code Generation

The code generators I've written always take two inputs: a UML diagram, and a code template.  The template is derived from an example of handwritten code.  That's the tricky part.  The code you templatize has be good, because you have to live with the resulting code.  Development of a new generator using the new stack has started as an iteration of what was learned with the existing one.

Code generators have been around for a long time, but adoption has been nearly nil.
Initially they were very expensive (IBM Rational Rose, Micro-Focus Together.) A major goal of these products is round trip coding and design.  That's difficult to obtain without other issues.  The idea that you could produce code, hand modify it, and reverse feed the modifications back into the design is not a realistic goal.  It leads to inflexibility in the type of code emission and sub-optimal code.  

"One Way Code Generation" is much simpler, and yields the following benefits:
  1. You can emit anything. SQL, Java, Python, C++, etc.
  2. The generator itself is fairly simple.
  3. Code templating is straight forward.

The hardest part of code generation is providing the optimal code to templatize. You have to really like the code, because you will see the template generated pattern repeated everywhere the template's used. 

Pattern.  That's a concept that shows up in the literature a lot.  The 'Gang of Four'.  Aside from the patterns discussed in the many patterns books, there are often patterns unique to a project.  The trick is in seeing them.  Once found, the next trick is in determining if the pattern is optimal, and fixing the pattern if it isn't.  The final step is discerning whether the pattern is fodder for a C++ template (which are really compile time code generators), a common function, or a code template for the generator.

Example Code Template for C++


  Object Definition for [[CLASSNAME]].
  Copyright (c) 2018 by D.K. McCombs.
  W3 Systems Inc.  

#ifndef [[U-CLASSNAME]]_H
#define [[U-CLASSNAME]]_H
#include "types.h"

using namespace std;

namespace w3sea {

class [[CLASSNAME]]

  [[TYPE]] [[FIELD]];


  // default constructor
  [[CLASSNAME]]() = default;
  // copy constructor
  [[CLASSNAME]](const [[CLASSNAME]] & in) = default;
  [[CLASSNAME]]([[CLASSNAME]] && in) = default;
  // assignment operator
  [[CLASSNAME]] & operator = (const [[CLASSNAME]] & in) = default;
  [[CLASSNAME]] & operator = ([[CLASSNAME]] & in) = default;
  // destructor  
  virtual ~[[CLASSNAME]]() = default;
  // access 
  [[TYPE]] get_[[FIELD]]();
  void set_[[FIELD]](const [[TYPE]] & in);




Applying The UML "session" class:

The code generator merges this UML class with the code template.

Generated Code


  Object Definition for session.
  Copyright (c) 2018 by D.K. McCombs.
  W3 Systems Inc.  

#ifndef SESSION_H
#define SESSION_H
#include "types.h"

using namespace std;

namespace w3sea {

class session

  identifier id;
  time_date created;
  time_date last_used;
  string user_agent;
  string remote_address;
  string nav;
  string uuid;
  string json;
  long long user;


  // default constructor
  session() = default;
  // copy constructor
  session(const session & in) = default;
  session(session && in) = default;
  // assignment operator
  session & operator = (const session & in) = default;
  session & operator = (session & in) = default;
  // destructor  
  virtual ~session() = default;
  // access 
  identifier get_id();
  void set_id(const identifier & in);

  time_date get_created();
  void set_created(const time_date & in);

  time_date get_last_used();
  void set_last_used(const time_date & in);

  string get_user_agent();
  void set_user_agent(const string & in);

  string get_remote_address();
  void set_remote_address(const string & in);

  string get_nav();
  void set_nav(const string & in);

  string get_uuid();
  void set_uuid(const string & in);

  string get_json();
  void set_json(const string & in);

  long long get_user();
  void set_user(const long long & in);



Generic Database Access Classes

This is a high level diagram of the generic class interface library for data access in the W3Sea:

The class interface library will have implementations for Postgresql, Mysql (or Mariadb), and Sqlite.  It should also be fairly easy to pickup DB2 and ODBC.

The way it works:

First instantiate a connection.  There will be a factory to create one based on the targeted database.

Next instantiate a statement.  Part of the instantiation of a statement is to pass an SQL command to the constructor.  The command can be any of the normal insert, update, query or delete commands according to the SQL specification. 
If there are values to insert or update, or if a 'where' clause exists in the command, use '?' markers for the actual parameters.  This mitigates the possibility of SSL injection.

Next bind the actual values to the '?'  markers using the statements 'bind()' method.

If the statement was a query, then a cursor is available to bind external variables to the returned row and columns.  You can check if the cursor is valid by calling the method 'is_valid()'.
You can bind the return variables all at once by calling the bind_fields() method of the cursor.  This method uses a variadic template.  You should do this before calling execute, because, in the case of a query, the execute automatically moves the cursor to the first row.

Next call execute.  Execute should return the success or failure as a boolean result.

In the case of update, insert, or delete, you can query for the number of rows effected by calling the statement rows_effected() method. 

In the case of a query, the bound values are already set.  This allows very easy binding of properties in a class object, without having to have a lot of intrusive code in the class.

Calling the cursor 'next() method moves to the next row, and again, as like 'execute()', the external bound values are automatically set to the new row values.

No need to close anything since the library uses the RAII pattern, which automatically closes things for you.  This is about as simple a database interface as I know to make. All the user needs to know is how to use a connection, a statement, and a cursor.

Javascript Libraries

The problem with Javascript libraries is that you have to pick one.  There are always winners and losers.  HTML5 Javascript and DOM is rich enough.  The compelling reasons for external libraries (missing features) has diminished.  Other than shim libraries for compatibility, most of the libraries benefit today devolves to shorthand interfaces to whats already there.  The HTML5 DOM is an internationally recognized standard.

For these reasons, the W3SEA javascript will use the HTML5 DOM directly instead of depending on other external libraries.  


I don't like the current state of 'mobile device'  detection.
Many of the methods I looked at use the "user_agent" HTTP header and are apt to fail over time.

My current method doesn't use user_agent — because the standards committees say it's wrong. 

The W3C or WHATWG really need to do some work on this.

I'm currently using the CSS3 selector:
screen and (max-width: 1080px)
This is a simple approach that will work until phone displays have 4K screens.
(Oh no! They already do!) 

I'll be hacking for quite a while on a future proof approach ...

The issue is the relative size of the fonts in comparison to the screen, a font size in points is supposed to be a measure in inches (1pt=1/72 inch) but they are not even close to that size on a mobile device. This is on purpose, because of the screen size. 

So a reasonable approach would be to measure the size of a word against some heuristic standard to determine if the font is too small.  It's not mobile detection we are really after, but point size v.s. screen size.
I don't want to have separate sites for mobile and PC.
(One site to rule them all.)