View Issue Details

IDProjectCategoryView StatusLast Update
0009250New Feature Requests[All Projects] Generalpublic2018-07-21 19:49
Reporterharon4iggAssigned To 
PrioritynormalSeverityfeatureReproducibilityN/A
Status newResolutionopen 
Summary0009250: [Request] Extended ElementData Sync
Description

Trying a lot of different techniques to sync data between server and clients i'm always facing bandwidth and cpu drops on server-side or fps-drops on clientside.

Summarizing my experience, i would like to suggest a feature, something like

setElementDataEx(element, key, value, syncType=normal, syncTargets = root, forceSync=true)
forceElementDataSync()

This function should work in conjunction with the classic elementData, but the difference is how things will be synchronized.

syncType:
1) private - only players listed in syncTargets will receive the update. if new player joins or reconnects the server, the date is hidden for him, but if same key and same value will be set targeting to this player, he should receive the update.

2) normal - direct update will receive only syncTargets, sets data as public, if new player join the server, he should receive this data as well.

3) stream - this data should be synchronized only when player streams the element or element already streamed in. syncTargets is ignored here, but can be used to filter players who should receive data when element streamed. When element streams out, the data should stay in memory, so next time only delta will be synced. (it's a little bit tricky, we need to store some timestamp of last sync, but it will save a bandwidth and cpu.)

forceSync:
if true - data should be directly sent like setElementData, if false - the MTA should push it into buffer and collect all changes until current scope ends or method forceElementDataSync() triggered. This means the data should be synchronized not like many single packets, but one batch operation, or can be splitted in to few buckets for better shipping and client-side processing. The order of operations should be keeped. If newly set value overlaps previously set one in pending buffer, it should be overwritten.

forceElementDataSync() - forces operations buffer to be synced.

p.s. i know that most of it can be done on lua side, but there is a few limitations with setElementData without sync, which forces you to keep a duplicated element data tree. Actually i already did most of it in lua, but some of operations causing FPS drops due to lua quickness, so i think it should be implemented closely to core :)

TagsNo tags attached.

Relationships

has duplicate 0008676 closedqaisjp New issues [Request] Add an option to sync element data only with a particular client 
has duplicate 0009830 closed New issues Add bSync and syncTo arguments to setElementData at client, and server-side. 
has duplicate 0009708 closedmyonlake New Feature Requests [Request] setElementData sync between selected player and server. 
related to 0007950 closed New issues [Request] setElementData option to synchronize to streamed in only clients 

Activities

arranTuna

2016-05-26 16:53

manager   ~~0024714

Related to and partially the same as #8676 and #7950

qaisjp

2016-05-26 18:23

administrator   ~~0024715

Last edited: 2016-05-26 18:31

View 5 revisions

This is an interesting suggestion.

On the Lua interface side, I think it would be simpler to just combine those two arguments into one arg:

  • giving "stream" will do as you say
  • giving a table will be private, and only sync to the players in the table
  • giving falsey will do what it currently does

You can get rid of that last argument and require the extra function call for a forceSync.

I can't say anything about whether the feature itself is possible and whether it should actually be implemented.

I imagine that the stream and forceSync functionality will be more difficult to implement.

Also - about private data - would this only send the data to clients and not store it on the server? Maybe this could be a security issue because the server cannot catch the update (it has no idea about it).

Finally - I also think it's important to evaluate the necessity of this function (see #7950, as Arran referenced). "Since the introduction of triggerClientEvent accepting a table of players there isn't the efficiency concern any more."

haron4igg

2016-05-26 19:58

viewer   ~~0024718

It is definitely difficult to implement but this will remove a lot of synchronisation bottlenecks and make everything more smoothly, which will give you possibility to build more interesting gamemodes. I have a lot of great ideas, but they requires a lot of data to be transferred and with current implementation of element data it is not possible to make it work fast and smoothly with minimal traffic consumptions.

Using triggerClientEvent not solving issues with performance, for example you need to transfer few values . what will you decide? use 3 separate calls, or send dictionary with changes? first one method will consume more traffic and cpu for establishing connections on server side and second one will drop FPS on client machine while lua will cycle over this values and apply it as private element data... and if you need to send also all the changes to newly joined players? You need to implement a lot of side logic to store values somewhere, to synchronize it later, using twice more lua memory. And if you need to use this system between separate resources? you will create separate store for this pending data for each resource? or use exports which are very sloooow when transfer more then zero parameters?


Private data means server know about this and some players know about this - when new player join the server - this data is not synced with him. For example. Player have few characteristics which is specific only for this player and his teammates. I need to update them - i triggering private update for whole team. other players don't know anything about it. The element data still can be used for client-server logic, when player reconnects i setting up one more time his private data from server side.


In other words i think that mta MUST have 3 features for element data to be perfect in synchronisation sence:
1) Control visibility of data between clients
2) Send multiple data during one transaction
3) Make synchronisation streamable.

I can implement such synchronisation by myself in java, swift, c,objective-c, but only for iOS and Android...(( i'm not familiar with windows platform at all(( and also not familiar with MTA inside... but i can help with brainstorming, stabilizing and testing :)

Lex128

2016-05-28 09:10

reporter   ~~0024725

This would be a very useful feature. And I think that syncType and syncTargets can be combined into one parameter

IIYAMA12

2016-05-28 16:50

viewer   ~~0024726

In my opinion the trigger event's are using too much network in comparison with setElementData. (7 t/m 10 times more for the same data?)

This would be a good alternative or the trigger event's should be optimised instead. If that isn't possible, then there should be a sort of 'share with server'/'share with client' function, without using the event system.

But don't forget reason why we have the function setElementData. It is simply for adding custom data to elements. The synchronization feature for disable sharing is handy, but because of that feature we consider elementData now as a communication function (for everything) between client and server. While it was only meant for adding something extra to an element. You might say that players are elements, that's correct, but they are also external computers. In my opinion 'element data'(the name and it's function) is not the way we should communicate all kinds of data between the client and server. There should be an alternative over elementData and triggerEvents, they both have their own functions, but they do more than simply share data with the server and a single client.

Pirulax

2018-03-25 15:36

reporter   ~~0026479

Definitely, it would be a good idea, but i think, for now a simple implementation of #0009830 would be enough, its not as tricky to implement as this, but would reduce lags caused by syncing.

Pirulax

2018-03-25 15:42

reporter   ~~0026480

And yes, just as @qaisjp said, settings syncType to false would cause to not sync that elementdata.

Saml1er

2018-03-25 16:53

updater   ~~0026481

I wish if it was possible to completely remove element datas. They ruin the code design. I used to find them helpful when creating small scripts, but I later learned how much better and efficient it is to use triggerClientEvent. It would be a complete waste of time, if someone tries to add more features to this, sorry but just no.

arranTuna

2018-03-25 17:53

manager   ~~0026482

If you don't like element data, then don't use it..? And you're also the only person who thinks that way, element data is the best way to do a lot of things, how the hell else would one script be able to know something that another script has set on an element without using exports which are inefficient and the export would be ruined if that script restarted.

Element data that syncs from client to server and other streamed in players would be very useful for some scripts like parachute at present the element data it uses is synced to all players with is unnecessary and also there's various scripts of my own that would benefit from this like my animation sync script which currently relies on triggerClientEvent to tell nearby players when another players animation changes, with synced element data this could be achieved all in the client side script. Also objects attached to players, custom skin shaders, custom clothing shaders, etc.

myonlake

2018-07-03 14:42

updater   ~~0026597

If this issue is worked on, it could be a good idea to also figure out client-side removeElementData while at it. I suppose the reason it's still todo is because of the synchronized data between client and server.

arranTuna

2018-07-03 20:15

manager   ~~0026600

AFAIK setElementData(element, "data", nil) does the same as removeElementData.

codylewiz

2018-07-03 20:16

viewer   ~~0026601

No it doesn't.

Pirulax

2018-07-08 21:53

reporter   ~~0026675

What if syncTo changes?
For ex, the first time I set the syncTo to root and then change it to a myself only.
Should I add a function like: setElementDataSyncTarget(element theElement, string key, element / table syncTo)?
setElementData would still contain syncType and syncTo.

Issue History

Date Modified Username Field Change