Storing objects in the Google Firebase cloud with TMS Cloud Pack

Tuesday, June 20, 2017

Since the latest release of TMS Cloud Pack, there is now also built-in support to use Google Firebase for cloud data storage. The architecture of Google's Firebase is quite simple. It offers storage of JSON data in the cloud. The JSON data has a unique identifier in a table and via access with this unique identifier, this JSON data can be read, updated or deleted. In addition, indexing rules can be set to perform query on the data in a Firebase table, for example, retrieve all JSON objects where a field X has value Y.

To make it really easy use to use data on Google Firebase from a Delphi or C++Builder application, we have added capabilities to put objects or generic lists of objects in a Firebase table. This is done via the non-visual component TFirebaseObjectDatabase that you can put on the form.



To demonstrate this, consider a class we want to use in the Delphi application that descends from TFirebaseObject:

  TFirebaseCustomer = class(TFirebaseObject)
  private
    FName: string;
    FStreet: string;
    FZIP: integer;
    FDoB: TDate;
    FCity: string;
  public
    constructor Create; override;
    destructor Destroy; override;
  published
    property Name: string read FName write FName;
    property Street: string read FStreet write FStreet;
    property City: string read FCity write FCity;
    property ZIP: integer read FZIP write FZIP;
    property DoB: TDate read FDoB write FDoB;
  end;

Now, after we have retrieved a connection for TAdvFirebaseObjectDataBase to Firebase via:

  AdvFirebaseObjectDatabase1.DatabaseName := 'TMS';
  AdvFirebaseObjectDatabase1.TableName := 'Customers';
  AdvFirebaseObjectDatabase1.Connect;
we can create and put TFirebaseCustomer objects in the Google Firebase realtime data cloud:

var
  cst: TFireBaseCustomer;
begin
  cst := TFireBaseCustomer.Create;
  try
    cst.Name := 'Bill Gates';
    cst.Street := 'Microsoft Av';
    cst.ZIP := 2123;
    cst.City := 'Redmond';
    cst.DoB := EncodeDate(1969,04,18);
    cst.ID := '1240';
    AdvFirebaseObjectDatabase1.InsertObject(cst);
  finally
    cst.Free;
  end;
end;

All published properties of the object will be automatically persisted on the Google Firebase cloud. This is how the data looks when inspecting it on the Firebase console

Note that we have explicitly set the unique ID of the object via the cst.ID property. When the ID is set at application level, it is the responsibility of the app to use unique IDs. When no ID is set, the AdvFirebaseObjectDatabase will automatically create a GUID as ID.

In this example, we have created a rather simple object with simple data types. But nothing prevents you from using class properties as in this example:

  TCareerPeriod = class(TPersistent)
  private
    FFinish: integer;
    FStart: integer;
  published
    property Start: integer read FStart write FStart;
    property Finish: integer read FFinish write FFinish;
  end;

  TFirebaseCustomer = class(TFirebaseObject)
  private
    FName: string;
    FStreet: string;
    FZIP: integer;
    FDoB: TDate;
    FCity: string;
    FPicture: TFireBasePicture;
    FCareer: TCareerPeriod;
    procedure SetCareer(const Value: TCareerPeriod);
  public
    constructor Create; override;
    destructor Destroy; override;
  published
    property Name: string read FName write FName;
    property Street: string read FStreet write FStreet;
    property City: string read FCity write FCity;
    property ZIP: integer read FZIP write FZIP;
    property DoB: TDate read FDoB write FDoB;
    property Career: TCareerPeriod read FCareer write SetCareer;
  end;

And now this code can be used to persist this slightly more complex object:
var
  cst: TFireBaseCustomer;
begin
  cst := TFireBaseCustomer.Create;
  try
    cst.Name := 'Elon Musk';
    cst.Street := '3500 Deer Creek Road';
    cst.ZIP := 2123;
    cst.City := 'Palo Alto';
    cst.DoB := EncodeDate(1975,03,21);
    cst.ID := '1241';
    cst.Career.Start := 2011;
    cst.Career.Finish := 2017;
    AdvFirebaseObjectDatabase1.InsertObject(cst);
  finally
    cst.Free;
  end;
end;

As you can see, more complex classes can be easily & automatically persisted in the Google Firebase cloud.


With respect to types of fields, there is one caveat though, Google Firebase doesn't offer out of the box support for binary blobs. Imagine that we'd want to persist a Delphi object that has a TPicture property. The TPicture is internally streamed in a custom way to the DFM (via the DefineProperties(Filer: TFiler); override) and the JSON persister does not automatically get this data. There is however an easy workaround to add a published string property to a class descending from TPicture and use this string to hold hex encoded binary data of the picture. The DataString property getter & setter methods use the StreamToHex() and HexToStream() functions that are included in the unit CloudCustomObjectFirebase:

  TFireBasePicture = class(TPicture)
  private
    function GetDataString: string;
    procedure SetDataString(const Value: string);
  published
    property DataString: string read GetDataString write SetDataString;
  end;


{ TFireBasePicture }

function TFireBasePicture.GetDataString: string;
var
  ms: TMemoryStream;
begin
  ms := TMemoryStream.Create;
  try
    SaveToStream(ms);
    Result := StreamToHex(ms);
  finally
    ms.Free;
  end;
end;

procedure TFireBasePicture.SetDataString(const Value: string);
var
  ms: TMemoryStream;
begin
  ms := HexToStream(Value);
  try
    LoadFromStream(ms);
  finally
    ms.Free;
  end;
end;


This way, we can extend the class to have a picture property:

  TFireBaseCustomer = class(TFirebaseObject)
  private
    FName: string;
    FStreet: string;
    FZIP: integer;
    FDoB: TDate;
    FCity: string;
    FPicture: TFireBasePicture;
    FCareer: TCareerPeriod;
    procedure SetPicture(const Value: TFireBasePicture);
    procedure SetCareer(const Value: TCareerPeriod);
  public
    constructor Create; override;
    destructor Destroy; override;
  published
    property Name: string read FName write FName;
    property Street: string read FStreet write FStreet;
    property City: string read FCity write FCity;
    property ZIP: integer read FZIP write FZIP;
    property DoB: TDate read FDoB write FDoB;
    property Career: TCareerPeriod read FCareer write SetCareer;
    property Picture: TFireBasePicture read FPicture write SetPicture;
  end;

and persist the object with picture via:

var
  cst: TFireBaseCustomer;
begin
  cst := TFireBaseCustomer.Create;
  try 
    cst.Name := 'Tim Cook';
    cst.Street := '1 Infinite Loop';
    cst.ZIP := 2123;
    cst.City := 'Cupertino';
    cst.DoB := EncodeDate(1975,03,21);
    cst.ID := '1242';
    cst.Picture.LoadFromFile('tim_cook.jpg');
    cst.Career.Start := 2006;
    cst.Career.Finish := 2017;

    AdvFirebaseObjectDatabase1.InsertObject(cst);
  finally
    cst.Free;
  end;
end;

To retrieve the objects back that were persisted, we can use:

var
  fbo: TFirebaseObject;
begin
  fbo := AdvFirebaseObjectDatabase1.ReadObject('1240');

  if Assigned(fbo) then
  begin
    if (fbo is TFirebaseCustomer) then
      ShowMessage((fbo as TFirebaseCustomer).Name+';'+(fbo as TFirebaseCustomer).City);
    fbo.Free;
  end;
end;



To persist a property change of the object, for example to change the career date of customer 1241 to 2018, we could write:

var
  fbo: TFirebaseObject;
begin
  fbo := AdvFirebaseObjectDatabase1.ReadObject('1241');

  if Assigned(fbo) then
  begin
    if (fbo is TFirebaseCustomer) then
    begin 
      ((fbo as TFirebaseCustomer).Career.Finish := 2018;
      AdvFirebaseObjectDatabase1.WriteObject(fbo);
    end;
  end;
end;

Working with generic lists


The previous samples all showed how to perform CRUD operations on single objects in the Google Firebase cloud. Our component TAdvFirebaseObjectDatabase also facilitates dealing with generic lists of objects. Imagine you want to persist a score table of game and have following simple classes to store this:

  TSimpleClass = class(TFirebaseObject)
  private
    FName: string;
    FScore: integer;
  published
    property Name: string read FName write FName;
    property Score: integer read FScore write FScore;
  end;

  TSimpleList = TList<TSimpleClass>;


With this class and its generic list, we can add a list of data at once in the Google Firebase cloud:

var
  sl: TFirebaseObjectList;
  sc: TSimpleClass;
  I: Integer;
begin
  sl := TFirebaseObjectList.Create;
  try
    sc := TSimpleClass.Create;
    sc.Name := 'Bruno';
    sc.Score := 48;
    sc.ID := '1';
    sl.Add(sc);

    sc := TSimpleClass.Create;
    sc.Name := 'Nancy';
    sc.Score := 83;
    sc.ID := '2';
    sl.Add(sc);

    sc := TSimpleClass.Create;
    sc.Name := 'Pieter';
    sc.Score := 17;
    sc.ID := '3';
    sl.Add(sc);

    sc := TSimpleClass.Create;
    sc.Name := 'Bart';
    sc.Score := 299;
    sc.ID := '4';
    sl.Add(sc);

    sc := TSimpleClass.Create;
    sc.Name := 'Wagner';
    sc.Score := 55;
    sc.ID := '5';
    sl.Add(sc);

    AdvFirebaseObjectDatabase1.TableName:= 'List';
    AdvFirebaseObjectDatabase1.InsertList(sl);

    for I := 0 to sl.Count - 1 do
      sl[I].Free;
  finally 
    sl.Free;
  end;
end;

Inspecting this on the Google Firebase console results in:



If at a later time we'd like to retrieve the score of Bart, we could use the code:

var
  c: TFirebaseObjectList;
  I: Integer;

begin
  AdvFirebaseObjectDatabase1.TableName := 'List';
  c:= AdvFirebaseObjectDatabase1.QueryList('Name', 'Bart');
  if Assigned(c) then
  begin
    for I := 0 to c.Count - 1 do
    begin
      ShowMessage((c[I] as TSimpleClass).Score.ToString);
      c[I].Free;
    end;
    c.Free;
  end;
end;

This performs a query on the List table for value 'Bart' set in the 'Name' field. The result of the query is a generic list (in case multiple matching results are found).

This was a glimpse at the TAdvFirebaseObjectDataBase component as an introduction for the new Google Firebase access features in the TMS Cloud Pack. Explore the full set of powerful capabilities of TAdvFirebaseObjectDataBase and the many other components in TMS Cloud Pack to make consuming cloud services in Delphi & C++Builder a piece of cake.


Bruno Fierens


Bookmarks: 

This blog post has received 7 comments. Add a comment.



TMS VCL Cloud Pack v3.7 introduces new level of seamlessness to access cloud data

Thursday, September 22, 2016

In TMS VCL Cloud Pack v3.7, new components TAdvmyCloudDataConnection and TAdvmyCloudDataDataSet have been added. With these 2 components, access to structured data on the cloud via the myCloudData service becomes easier than ever.

TAdvmyCloudDataConnection
TAdvmyCloudDataConnection is a non-visual component that manages access to myCloudData. It works as the intermediator for the authentication and authorization to access myCloudData for one or multiple TAdvmyCloudDataDataSet instances. This means that via the TAdvmyCloudDataConnection at least a one-time authentication and authorization is done with myCloudData to obtain the access token and the TAdvmyCloudDataDataSet can then work through TAdvmyCloudDataConnection to use this access token. To let the TAdvmyCloudDataDataSet use the connection is as simple as assigning the TAdvmyCloudDataConnection instance to TAdvmyCloudDataDataSet.Connection, just like we are used to assign a TADOConnection to a TADOTable for example.


TAdvmyCloudDataDataSet
TAdvmyCloudDataDataSet is a wrapper as a dataset of a table in the myCloudData service. When a connection is provided, set either the name of the table via the Table property or the unique ID of the table via the TableID property. When a connection has been made (i.e. access token was obtained), setting the property TAdvmyCloudDataDataSet.Active to true will fetch the table data from the myCloudData table and present it as a TDataSet through a TDataSource to any DB-aware component in VCL or via LiveBindings in FireMonkey applications. Note that this approach works both at run-time and design-time, so we can work with the cloud data at design-time to configure our DB-aware controls connected to it.


Dataset filter
An easy way is provided to perform basic dataset filtering. The TAdvmyCloudDataDataSet has a filter property which is presented as a collection of filter conditions. The filter condition consists of a field name, filter value, a filter condition logical operator and a filter comparison operator. The filter condition can be easily set at design-time where the dataset field can be picked and the values set.



From here we can work with the cloud data in much the same way from our code on the dataset as Delphi developers have been doing all the time with datasets, i.e. insert, delete, update records.

Rich metadata
But with the updated myCloudData service, there is much more than this possible as myCloudData now offers rich metadata for its structured data. This rich metadata can be programmatically accessed, created and updated but also easily configured via the myCloudData web interface. We'll cover this in a follow-up blog focused on how we can benefit from this rich metadata, but here you can already glance over the web interface you can access from your myCloudData account:


You can explore all this with a free myCloudData account. Create a new table via this account and you can use our TAdvmyCloudDataDataSetDemo application included in TMS VCL Cloud Pack to explore this.

Stay tuned for more!

Bruno Fierens


Bookmarks: 

This blog post has received 2 comments. Add a comment.



Weekend fun with Raspberry Pi 2 and TMS Cloud Pack

Sunday, June 07, 2015

Thinking to myself, why go through the hassle of sacrificing a desktop computer or fiddle around with VMs to test something on a Linux machine when I have here several 85mmx56mm Raspberry Pi boards laying around capable of amazing things. For my experiments, I wanted to have an ownCloud available and first thing to do was getting an ownCloud up and running on the Raspberry Pi 2, which turned out to be a piece of cake. To do this, follow these instructions:

Step 1: getting Apache up and running

If Apache is already setup on your device, skip this step, otherwise, from the command line, execute:

$ sudo apt-get install apache2

The Raspberry Pi 2 LED flashes for a short while and when completed, verify it is working by opening a browser on a machine in the network and navigate to the IP address of the Raspberry Pi 2 like http://192.168.1.100. When install was successful, you'll be greeted with an "It works!" page.

Step 2: install PHP and tools

In case you had Apache already configured with PHP 5, you can also skip this step.
To install, execute following commands:

$ sudo apt-get install php5
$ sudo apt-get install php5-gd
$ sudo apt-get install sqlite
$ sudo apt-get install php5-sqlite
$ sudo apt-get install php5-curl

After some more LED flickering, these steps will also be executed and the Apache environment with PHP 5 is ready.

Step 3: install ownCloud

To install ownCloud, start by downloading the latest distribution. At this time, this is v8.0.3 and is downloaded with:

$ sudo wget https://download.owncloud.org/community/owncloud-8.0.3.tar.bz2

Next step is to unpack the ownCloud distribution and install it under Apache.

$ sudo mv owncloud-8.0.3.tar.bz2 /var/www
$ cd /var/www
$ sudo bunzip2 owncloud-8.0.3.tar.bz2
$ sudo tar xf owncloud-8.0.3.tar

Step 4: setting up ownCloud

To finalize, the data folder for ownCloud must be created and an admin account added.
The data folder is created with following steps:

$ sudo mkdir /var/www/owncloud/data
$ sudo chown www-data:www-data /var/www/owncloud/data
$ sudo chmod 750 /var/www/owncloud/data

At this time it is more convenient to continue the setup from a browser. Either start the graphical shell on the Raspberry Pi 2 or connect from a browser on an external machine and navigate to http://localhost/owncloud or from external machine http://192.168.1.100/owncloud/
The first screen that appears is to create an admin account, so add the credentials for an admin account on this page. With this account added, owncloud is up and running and ready for use.

Step 5: Connecting from Delphi to ownCloud

Now Delphi kicks in and with a little help from the TMS Cloud Pack, let's start using ownCloud from a Delphi app.

Start your IDE, make sure TMS Cloud Pack is installed and drop the component TAdvCalDAV on the form. Configure the TAdvCalDAV component to access your ownCloud. Verify the ownCloud primary CalDAV address by opening the calendar app within ownCloud via: http://192.168.1.100/owncloud/index.php/apps/calendar/ and in the bottom left corner, click on settings where this primary address is shown. By default, this should be:

http://192.168.1.100/owncloud/remote.php/caldav/

So, now we can configure the TAdvCalDAV component to connect to ownCloud on Raspberry Pi 2:

try
  AdvCalDav1.URL := 'http://192.168.1.100/owncloud/remote.php/caldav/';
  AdvCalDav1.Username := 'tms';
  AdvCalDav1.Password := 'tmsrocks!';
  AdvCalDav1.Active := true;
except
   Exit;
end;

After a successful connect, the calendars and their events available on ownCloud can be retrieved, here by filling the info in a listview:

var
  i: integer;
  cdi: TCalDavItem;
  li: TListItem;
begin
  // get events
  for i := 0 to AdvCalDav1.Items.Count - 1 do
  begin
    cdi := AdvCalDav1.Items[i];

    if cdi.vCalendar.vEvents.Count > 0 then
    begin
      li := ListView1.Items.Add;
      li.Caption := cdi.vCalendar.vEvents[0].Summary;
      li.SubItems.Add(FormatDateTime('dd/mm/yyyy hh:nn',cdi.vCalendar.vEvents[0].DTStart));
      li.SubItems.Add(FormatDateTime('dd/mm/yyyy hh:nn',cdi.vCalendar.vEvents[0].DTEnd));
      li.SubItems.Add(cdi.vCalendar.vEvents[0].Location);
      li.SubItems.Add(cdi.vCalendar.vEvents[0].Description.Text);
      li.Data := cdi;     
    end;
  end;
end;

Adding a new calendar item is equally easy:

var
  cdi: TCalDavItem;
  li: TListItem;
begin
  // set the calendar for the event
  cdi := AdvCalDav1.Items.Insert('Personal');  // add item to the "Personal" calendar
  cdi.vCalendar.vEvents.Add;
  cdi.vCalendar.vEvents[0].Summary := 'Schloss Dyck Classic Days';
  cdi.vCalendar.vEvents[0].Location := 'Jüchen, Deutschland';
  cdi.vCalendar.vEvents[0].Description.Text := 'Automobile Kulturgeschichte auf der Museums-Insel';
  cdi.vCalendar.vEvents[0].DTStart := EncodeDate(2015,7,31);
  cdi.vCalendar.vEvents[0].DTEnd := EncodeDate(2015,8;2);
  cdi.Post;
end;

or modifying an existing item

var
  cdi: TCalDavItem;
  li: TListItem;
begin
  cdi := ListView1.Selected.Data;
  cdi.vCalendar.vEvents[0].Summary := 'Schloss Dyck Classic Days 2015';
  cdi.Update;
end;

Similar to accessing the calendars of ownCloud, you can also access the contacts with the TAdvCardDAV component in a very similar way. And this rounded up my little weekend fun experiment. With so much ubiquitous computing power around and wonderful technology, aren't we living in very exciting times?

Bruno Fierens


Bookmarks: 

This blog post has received 2 comments. Add a comment.



Chrome-style application setting persistence/synchronisation with DropBox

Monday, April 08, 2013

Google Chrome has the very interesting feature to be able to store its settings via your Google account. This means that when you install Chrome on a different machine and associate it with your Google account, it will automatically "inherit" all settings of your other configs. Not only Google Chrome does this but increasingly Windows desktop applications and tools use online storage to offer the convenience of having identical configurations on multiple machines. Another excellent example is the Google Chrome extension Speed Dial 2 that can synchronize its settings among machines this way.

Now, nothing prevents us from doing the same for a Delphi application and with the TMS Cloud Pack, it becomes very simple to use a cloud storage service such as DropBox to allow the user to persist his settings in a DropBox folder and have these settings synchronised between different machines this way.

We have created a very rudimentary example to demonstrate the concept. The settings from the sample application are simply the contents of a listbox where items can be added or removed via the application. We save the settings as a simple text file and load this at application startup time from a DropBox account and save it back to DropBox when the application closes.



All we need to do is drop an instance of TAdvDropBox from the TMS Cloud Pack on the form, set the DropBox application key and secret (that can be obtained for free after registering with DropBox) load the access tokens and when the access tokens do not yet exist, get an access token for DropBox via an authentication/authorization step and call one line of code to download the settings file.
When the application closes, we simply upload the settings again with one call.
When we save the access token (here for reasons of simplicity of the demo in an INI file), this one time authentication/authorization with the DropBox account by the user is sufficient (which is similar for Google Chrome settings synchronisation too by the way)
The code with information in comments for application startup becomes:
procedure TForm1.FormCreate(Sender: TObject);
var
  acc: boolean;
begin
  Dirty := false;
  // set the DropBox application key & secret here that is provided by DropBox
  // for free when registering via: https://www.dropbox.com/developers/apps
  AdvDropBox1.App.Key := DropBoxAppkey;
  AdvDropBox1.App.Secret := DropBoxAppSecret;
  // Use simple INI file storage for the access token that DropBox will give
  AdvDropBox1.PersistTokens.Location := plIniFile;
  AdvDropBox1.PersistTokens.Key := '.sync.ini';
  AdvDropBox1.PersistTokens.Section := 'DropBox';

  if AdvDropBox1.App.Key <> '' then
  begin
    // Try to load an access token if it was already retrieved earlier
    AdvDropBox1.LoadTokens;
    // Test if the token is working
    acc := AdvDropBox1.TestTokens;
    if not acc then
      // If the token was not working try to refresh it
      acc := AdvDropBox1.RefreshAccess;

    if not acc then
    // No token was found or existing token is not valid, so authenticate/authorize via DropBox
      AdvDropBox1.DoAuth
    else
    // Download the settings from DropBox and apply
      LoadSettings;
  end;
end;
When the application is first used, there is no access token to download a file from the users DropBox account and in this condition, first the DropBox authentication login screen is shown:


followed by the authorization screen:


When the access token is obtained the first time, the TAdvDropBox component triggers OnReceivedAccessToken from where the token is first saved and then the settings downloaded and applied:
procedure TForm1.AdvDropBox1ReceivedAccessToken(Sender: TObject);
begin
  AdvDropBox1.SaveTokens;
  LoadSettings;
end;

When the application closes, we can simply save the settings in the Form's OnClose event via:
procedure TForm4.FormClose(Sender: TObject; var Action: TCloseAction);
begin
  listbox1.Items.SaveToFile(GetSettingsFileName);

  if Dirty and AdvDropBox1.TestTokens then
    AdvDropBox1.Upload(nil,GetSettingsFileName);
end;

The full source of the sample can be download here. With the TMS Cloud Pack offering similar access also to Microsoft Skydrive and Google Drive, it becomes very easy to change to the cloud storage of your preference by swapping to the component TAdvGDrive or TAdvSkyDrive.

Bruno Fierens


Bookmarks: 

This blog post has received 3 comments. Add a comment.




Previous  |  Next  |  Index