Some thoughts about modularization

Just for the record, this poses a problem about the (few) cases where the gui must access a domains' files (to choose the webroot for example). For now, the virtual file system class creates a temporary FTP user and uses it to access the domain's files. Nevertheless, this can not be done if there's the "round-trip" of having to call the backend to create such user. Therefore, if we get to the point of creating ftp users trhough the backend, we must also implement a "permanently hidden for-gui-usage-only ftp user" that is created along with the domain and never gets deleted.

Quote from joximu

- the i-MSCP Gui only reads&writes to the i-mscp database and does not need full MySQL root privileges

We need to add more functionality to daemon in order to bypass full root privs. i.e. some way to ask the daemon "Does this database exist?" before we schedule it.

Quote from kilburn

For now, the virtual file system class creates a temporary FTP user and uses it to access the domain's files. Nevertheless, this can not be done if there's the "round-trip" of having to call the backend to create such user.

I ran into this issue when doing net2ftp integration, we need some kind of gui accessible FTP. (scp/sftp with pub key auth maybe?)

Hello friends ,

First, sorry for the delay:

I post here a little discussion that is linked to the both Joximu and kassah proposals:

<nuxwin> I'm very tired with the current version
<nuxwin> very shit
<nuxwin> we will adopt joximu idea to start new version from scratch
<nuxwin> to resume
<nuxwin> data are added in GUI database but
<sci2tech> engine do shit, gui just write in database?
<nuxwin> GUI do nothing more
<nuxwin> yes
<nuxwin> with event handler for queue job
<nuxwin> sound like scheduled tasks for each servers
<sci2tech> ok from me. but there are some problems with this ftp as sql management. but can be done
<nuxwin> I explain to you, imagine the following:
<nuxwin> I've only GUI and database on one server
<nuxwin> on other server, I've services such as Apache, php, ftp..
<nuxwin> and on other, I've Database server for customers
<nuxwin> and on other, I've mail service ....
<nuxwin> Imagine now I adds a mail account from the GUI
<nuxwin> then, the GUI will just adds the relevant data in GUI in normalized way
<nuxwin> and will add ADD_MAIL event in queue
<nuxwin> and then, he will notify all servers
<nuxwin> then, all servers will look at the queue to see if he have awaiting task
<nuxwin> and do it if yes
<nuxwin> So, what you think about that ?
<nuxwin> Hard to explain but I think you got the idea
<nuxwin> so on, for example, the mail server will run this own engine part for mail
<sci2tech> yes
<nuxwin> take the data in the database
<nuxwin> and then, will add the account data like he want
<nuxwin> About customer Database now
<nuxwin> the GUI will only add relevant data for new database and user in normalized way too
<nuxwin> but will not create the database and user
<nuxwin> instead of that, he will adds event in queue
<nuxwin> such as ADD_SQLD and will notify the servers for event
<nuxwin> and then, the server for database will do the job
<nuxwin> but ok, still the problem of SQL users linked to the databases
<nuxwin> BTW, we should fin a way to link user to customers databases
<nuxwin> but you got the idea ?
<nuxwin> by doing like this, for example, if one user want not use the daemon at all, h ecan!
<nuxwin>I mean, he can just add cron task on each servers to look at event queue sometime, and then do the job if one is scheduled...
<nuxwin> you got the idea ?
<sci2tech> yes
<sci2tech> but for service
<nuxwin> what you don't understand ?
<sci2tech> Mysql will be same for all servers?
<nuxwin> in fact Mysql for the ispCP GUI database yes
<nuxwin> but with the idea we can have more than one mysql server for customer's databases and also, service such as Proftpd can have this own database on main Mysql server or another server..
<nuxwin> but right all services server will take data and see queue from same SQL server
<nuxwin> btw: (The Mysql server that host the GUI database)
<sci2tech> if all servers take data from same server for ftp
<nuxwin> what about ftp
<sci2tech> ftp read data directly from db for connection
<nuxwin> ok I explain
<nuxwin> for ftp
<sci2tech> why wait engine to add this data when GUI can do it faster?
<nuxwin> because with that user can use easily other ftp server than proftpd
<nuxwin> I explain
<nuxwin> 1. The GUI will only store username and password and mount_point for ftp account in normalized way
<nuxwin> and will add event such as ADD_FTP in queue
<nuxwin> then, the server in charge of FTP service will look at the queue and see a task for him and then, he will launch this own script to adding ftp account
<nuxwin> he will transform data FROM GUI like he want. He can store it in flat file or in specific database (like he want...). That allow also to use other Ftp server than proftpd as long plugin for engine is provided. Same concept for all other services sucha as postfix, courier....
<sci2tech> ah ok
<sci2tech> right
<sci2tech> this way we can use server that do not use Mysql
<nuxwin> yes
<sci2tech> this way we can use ftp server that do not use Mysql but plain file
<sci2tech> ok agree with that
<nuxwin> yes
<sci2tech> also we can split databases over multiple Mysql servers
<nuxwin> yes
<sci2tech> ok. joximu proposal has my support
<nuxwin> hehe
<nuxwin> Will report our discussion
<nuxwin> in the forum to show joxi
<sci2tech> great
<nuxwin> we have understand
<nuxwin> I've only add the kassah event handler proposal here
<sci2tech> great
<sci2tech> love this 2 guys
<nuxwin> yeah

I mostly agree with nuxwin's proposal. Nevertheless, I feel there's a missing piece in there.

Essentially, what happens if there are multiple servers that have to act on an "add mail" task in the queue? For instance, consider a situation where someone has:

(1) A mail gateway that does spam filtering
(2) A mail server that stores mails for end users (offering pop3/imap services)

When a new mail account is added, both servers have to update its configuration to enable the new e-mail address. Now imagine that one of the servers succeeds in adding this new mail account whereas the other one fails. How do they both report their success/failure to a single task? The gui database is doomed to only show one of their statuses.

Here's where "transaction based" tasks kick in. Essentially, there should be some "task dispatcher" who knows that both server A and server B should update their configs in order for the new e-mail address to become functional. Then, this "task dispatcher" would be the responsible for launching a "transaction" that comprises both tasks (updating the configs on server 1 *and* updating the configs on server 2). Obviously, the actual tasks (that compose the transaction) can be also stored in the database just like in nuxwin's proposal. Then, each server would report its success and failure respectively. Thereafter, the "task dispatcher" would know that one of the servers has failed, and would tell the other one to "rollback", while updating the "add e-mail transaction" to a failed status.

This way we could keep track of single operations that affect multiple servers at once, while maintaining a "stable" state when something fails. IMHO this is a basic requirement for a real multi-server control panel...

Comments?

Well we already have the issue kilburn is trying to solve, instead of multiple servers we have multiple config. For instance, when a domain is added, it has to be added to apache, mail and dns. If any one of those fail, we currently just get stuck. (Doesn't happen often thankfully)