Once a day, the system is abnormally loaded for 2-4 hours, whatever it may be
During the month, almost every day, at different times, but often in the morning, the system is abnormally loaded.
Hoster said that the load is created by the database
He also said that the fact that the panel shows more than 100% load on the processor is some kind of bug, it cannot be.
The tariff on the server was increased, but it did not help globally.
The system slows down a lot during this period, but it works.
What could it be (we thought it was possible that the outlet was updating its directories, turned off automation in the directories + turned off the update of the outlet categories, but nothing helped)?
Original question is available on version: ua
Hoster said that the load is created by the database
He also said that the fact that the panel shows more than 100% load on the processor is some kind of bug, it cannot be.
The tariff on the server was increased, but it did not help globally.
The system slows down a lot during this period, but it works.
What could it be (we thought it was possible that the outlet was updating its directories, turned off automation in the directories + turned off the update of the outlet categories, but nothing helped)?
Answers:
Good afternoon.
1. The load on the cpu can be more than 100%. This means that the tasks to be completed per second are X% more than the processor can handle. For example, if the processor on the server can execute 1000 tasks per second, and there are 1500 in the queue, then you will have a cpu load of 150% at that moment. It if it is direct strongly-strongly to simplify.
2. Most likely, some resource-intensive task is running on the server at this time. If you have a product version - os, we can analyze the load at the specified time and give advice on how to deal with it, it will take about 3 hours of completion.
1. The load on the cpu can be more than 100%. This means that the tasks to be completed per second are X% more than the processor can handle. For example, if the processor on the server can execute 1000 tasks per second, and there are 1500 in the queue, then you will have a cpu load of 150% at that moment. It if it is direct strongly-strongly to simplify.
2. Most likely, some resource-intensive task is running on the server at this time. If you have a product version - os, we can analyze the load at the specified time and give advice on how to deal with it, it will take about 3 hours of completion.
27.02.2023, 13:15
Original comment available on version: ru
1. And somehow it is possible to understand, at least theoretically, what kind of task it could be (if such obvious, complex ones were not presented), that is, are there any such tasks that the system regularly performs once a day, that is, any system tasks?
2. If on the server through the TOP command I show that the processor is loading the database, is it possible to somehow understand what this could be (what kind of task or which database table is the processor working with)?
2. If on the server through the TOP command I show that the processor is loading the database, is it possible to somehow understand what this could be (what kind of task or which database table is the processor working with)?
27.02.2023, 13:27
Original comment available on version: ua
Yulia wrote:
And somehow it is possible to understand, at least theoretically, what kind of task it could be (if such obvious, complex ones were not presented), that is, are there any such tasks that the system regularly performs once a day, that is, some kind of system tasks?
by default, the system does not have complex processes that load the system like that, there are tasks for transferring tasks, TTN tracking and other tasks once a day/time, but they cannot load the processor like that.
Yulia wrote:
2. If on the server through the TOP command I show that the processor is loading the database, is it possible to somehow understand what this could be (what kind of task or which database table is the processor working with)?
yes, at the same time, you need to look at what heavy requests are hanging in the database, you can use them to understand which module makes them and move on.
27.02.2023, 13:29
Original comment available on version: ua
.dev
OneBox production wrote:
yes, you need to watch at the same moment what heavy queries are hanging in the database, you can use them to understand which module is doing them and already move on.
And where can you marvel at the "important questions"?
27.02.2023, 13:41
Original comment available on version: ru
in the database itself at the time of loading, in the logs of heavy queries on the melon database. Unfortunately, I can’t tell you more specifically, since I don’t teach database administration and catching load causes.
27.02.2023, 13:44
Original comment available on version: ru
1. Is it normal (according to the screenshot) that the load increases only on the processor, but for example, the RAM does not increase?
2. Can some "google search robots" or something similar give similar loads?
2. Can some "google search robots" or something similar give similar loads?
27.02.2023, 15:19
Original comment available on version: ua
1. quite
2. If you have a website, then yes - Google can index it.
2. If you have a website, then yes - Google can index it.
27.02.2023, 15:53
Original comment available on version: ru
you can also be "parsed" by some other services.
27.02.2023, 15:53
Original comment available on version: ru
I am attaching statistics from phpMyAdmin
According to statistics, 3,275,000 select requests per hour
That is, it turns out that 3,275,000 / 60 = 54583 (per minute) / 60 = 909 (per second)
It turns out that there are almost 1000 requests per second
Somehow it's too much
According to statistics, 3,275,000 select requests per hour
That is, it turns out that 3,275,000 / 60 = 54583 (per minute) / 60 = 909 (per second)
It turns out that there are almost 1000 requests per second
Somehow it's too much
06.03.2023, 10:23
Original comment available on version: ua
Julia wrote:
Like it's a lot of fun
it depends on what you compare it to. It all depends on the complexity of the requests, for example, my PC makes 100,000 entries in the database in 10 seconds and at the same time loads the cores by less than 15%. Those. 10,000 records per second. Readings are much easier than writing.
If this is a problem for your server, I can only solve it if you have os. Unfortunately, I can't do anything with the mvp code, this version of the product is not supported. You can debug complex queries yourself, find them and rewrite them, or do any other optimization of your server. You can start by disabling the "once an hour" actions, for example, it may load price recalculation or something else. Also try to look in the server request log in nginx, maybe there will be requests for the pages of your site and someone is parsing you, so you can add his ip for example. In general, there are thousands of options for why your server is stupid and all of them are solvable.
06.03.2023, 20:47
Original comment available on version: ru
.dev
OneBox production wrote:
my pc makes 100,000 entries to the database in 10 seconds
You write about "records" (that is, adding a new row to the table) this is INSERT
And the statistics seem to show "requests", that is, the number of SELECTs per second
It's like slightly different things, maybe you meant that you have 10,000 SELECTs per second
It's just that in our case (there are approximately 3,000 active products available), it is not clear that it is possible to select so many times with SELECTs, that is, we have 3,000, and the system makes an average of 1,000 requests per second, and this is for 2-4 hours, that is, it is somehow unclear that it is possible parsing 3000 products with 3,275,000 requests per hour, it's more like some kind of DDoS attack
07.03.2023, 10:44
Original comment available on version: ua
After a little monitoring, you can see that periodically there is a load of 3000-4000 requests per second, this is with normal processor performance, it seems that during the peak of the load these requests occur every second and then there is a heavy load.
Looking at the size of the tables, there is a table of additional fields "shopcustomfield" it has approximately 3,100,000 records and a size of 1GB
Perhaps such an indicator of the "shopcustomfield" table creates such a number of requests and such a problem, or are these approximately normal indicators for this table?
Looking at the size of the tables, there is a table of additional fields "shopcustomfield" it has approximately 3,100,000 records and a size of 1GB
Perhaps such an indicator of the "shopcustomfield" table creates such a number of requests and such a problem, or are these approximately normal indicators for this table?
07.03.2023, 11:04
Original comment available on version: ua
Julia wrote:
And the statistics now show "request" that the number of SELECT per second
SELECT in most cases is several times lighter in terms of load for the database than INSERT, so I gave you an example with INSERT. Those. if I do 10k inserts, I can easily do that many reads.
Julia wrote:
Is it possible to have such a "shopcustomfield" table display and create such a number of requests and such a problem, what is approximately normal display for this table?
this is a relatively normal amount of data
I see your graphs, but unfortunately I can only say that they are very beautiful, there is no more valuable information for me in them. Yes, you have relatively many requests that it is not clear who, when and for what purpose makes. I wrote the solutions above, if you have os (apparently this is not the case), give a link to your box and I will help you. If you have mvp - debugging queries to the database is completely on your shoulders.
07.03.2023, 12:55
Original comment available on version: ru
.dev
OneBox production wrote:
In most cases, SELECT is several times lighter in load for the database than INSERT, so I gave you an example with INSERT. That is, if I'm doing 10k inserts, I can easily do the same number of reads.
You are talking about 10,000 added records
And the statistics show 1,000 requests (that is, for example, one request can select/add/update 10,000 records and it will be 1,000 * 10,000 in a second, these are slightly different things).
Let's try to optimize the database first
Based on the database analysis, it is clear that there are problems
For example, in the additional fields of the process products, there is a checkmark that we use "Automatically fill from the product filter"
So it turns out that we have, for example, 50 additional fields of process products that we use in one of the 20 processes, but do not use in the others
But it turns out that the system based on this tick "Automatically fill from the product filter" fills the fields for all BPs where products are added and as a result the number of entries in the table increases many times, that is, instead of 1 entry, there are 20 as a result (if the product was added to all 20 BP).
Perhaps there is a solution how to make this check box "Automatically fill from the product filter" work only on a specific BP, or maybe there is an action in the BP that will copy the product filters to the process products (that is, the reverse action of the action "Add a product filter from additional process fields") or maybe there is some action that can remove additional fields of process products?
07.03.2023, 14:01
Original comment available on version: ua
Julia wrote:
You are talking about 10,000 additional records
And the statistics show 1000 entries (for example, one entry can select/add/update 10,000 entries and it will be 1000 * 10,000 times three different words per second).
You didn't understand what I mean. Consider that I did not write anything to you about the number of requests.
Julia wrote:
Ale go scho the system on the basis of the cієї checkbox "Automatically load the filter of the product" will fill the fields according to all the BP where the goods are added and as a result, the number of entries in the tables in the times, to replace 1 record, їх in the result of the staє 20 (yakshcho the goods were added to all 20 BP).
You didn't start there. YOU DO NOT KNOW WHAT REQUESTS GO TO THE DB. But at the same time, you start to assume something and reduce some tables, which may not go requests at all. You need to find out:
1.What kind of requests
2. Where
3. How much
go to the database and then decide what to do with these requests. Not vice versa. If you do not have a programmer / system administrator who can conduct this analysis, you can not waste your time, you will not find anything. The fact that you started poking into random tables and making some assumptions may or may not fix the problem, you have about 300 + - attempts to "fix" some tables, as well as the number of tables. Will you touch everyone like that? A friend, all these 1000 requests per second go to the novaposhtainvoice table and do not depend on the number of data there?
07.03.2023, 14:10
Original comment available on version: ru
.dev
OneBox production wrote:
That's not where you started. YOU DON'T KNOW WHICH CLAIMS GO INTO THE DB. But at the same time, you start to assume something and reduce some tables, which you may not go into at all
Thank you, there is such a thing, you go to the store for bread, you come out with a package of other goods :)
Let's try to start with that.
Can you suggest a solution to the problem below?
For example, in the additional fields of the process products, there is a checkmark that we use "Automatically fill from the product filter"
So it turns out that we have, for example, 50 additional fields of process products that we use in one of the 20 processes, but do not use in the others
But it turns out that the system based on this tick "Automatically fill from the product filter" fills the fields for all BPs where products are added and as a result the number of entries in the table increases many times, that is, instead of 1 entry, there are 20 as a result (if the product was added to all 20 BP).
Perhaps there is a solution how to make this check box "Automatically fill from the product filter" work only on a specific BP, or maybe there is an action in the BP that will copy the product filters to the process products (that is, the reverse action of the action "Add a product filter from additional process fields") or maybe there is some action that can remove additional fields of process products?
09.03.2023, 14:35
Original comment available on version: ua
the number of selects does not depend on the number of values in the shopcustomfield, i.e. you are digging in the wrong place
09.03.2023, 14:38
Original comment available on version: ru
.dev
OneBox production wrote:
kvo select does not depend on kvo values in shopcustomfield, i.e. you are digging in the wrong place
The question is mostly not related to the main problem, I just would like to solve it as well, because it essentially affects the volume of the database and, as a result, the operation of the server as a whole (that is, the database is growing and, as a result, it needs a better server )
Can you suggest any solution to solve the described problem?
09.03.2023, 14:57
Original comment available on version: ua
Yulia wrote:
Can you suggest any solution to solve the described problem?
If you have mvp, I can't give you any solution. If you switch to os, I can provide it.
09.03.2023, 14:59
Original comment available on version: ua
.dev
OneBox production wrote:
If you switch to os, I can provide it.
Regarding the transition, we are now in a state of decision-making (there are a number of transition problems, including the website)
Currently, there is no solution on the OS either, do you need to make improvements?
09.03.2023, 15:47
Original comment available on version: ua
it will be possible to make a setting so that it does not fill in the value if it is not in the filter. Those. so that it does not generate empty records, it will take a couple of hours.
09.03.2023, 16:38
Original comment available on version: ru
.dev
OneBox production wrote:
can be configured to not populate a value if it is not in the filter. That is, so that it does not generate empty records, it will take several hours.
Here the question is not empty, but the fact that the system now fills for all BPs to which the product is added, and we only need, for example, for one BP
09.03.2023, 16:45
Original comment available on version: ua
there is an opportunity to choose a BP, if the BP is not the one that is selected, then we will not substitute
09.03.2023, 17:13
Original comment available on version: ru
.dev
OneBox production wrote:
there is an opportunity to choose a PSU, if the PSU is not the one selected, we will not replace it
Ok, thanks, let's look this way
09.03.2023, 17:52
Original comment available on version: ua
.dev
OneBox production wrote:
yes, at the same time, you need to look at what heavy requests are hanging in the database, you can use them to understand which module makes them and move on.
Good day
We saw in the database that the system sends many requests of the following format: "SELECT * FROM `novaposhtacontragentcontact` USE INDEX (index_refplatformid) WHERE `counterpartyRef`=........."
We saw that in the samples, the system selects data on clients with whom they have been associated for a year or more (that is, it selects some irrelevant data, that is, it seems that it sorts through all the data)
Immediately after the selection, there are requests of the following format: "UPDATE `novaposhtacontragentcontact` SET `CounterpartyProperty`='Recipient', `Description`= ......"
And this is done regularly.
Approximately 12,000 such requests are issued in a few seconds
Accordingly, there is a load on the processor.
Can you tell me why the system makes so many requests regularly for quite old data and what can be done to prevent the system from doing this?
14.06.2023, 11:41
Original comment available on version: ua
12k update requests to a table with 12k records will not give any load.
This is an update of the list of senders / recipients from the new mail.
This is an update of the list of senders / recipients from the new mail.
14.06.2023, 13:32
Original comment available on version: ru
.dev
OneBox production wrote:
12k update requests to a table with 12k records will not give any load.
This is an update of the sender/receiver counterparty list from the new mail.
It all depends on the server parameters + on other loads
And it turns out that there are other loads + these and as a result everything hangs
Can you tell me why the system updates all records of all clients and does it not at night, but regularly (perhaps every 5 minutes)?
It's just not normal from the point of view of rationality (regularly try to update data on all clients, at least you can update only those clients that have had changes and do it not so often and at night)
14.06.2023, 13:48
Original comment available on version: ua
Yulia wrote:
(perhaps every 5 minutes)
once in a while maximum
Yulia wrote:
It's just not normal from the point of view of rationality (regularly try to update data on all clients, at least you can update only those clients that have had changes and do it not so often and at night)
Well, if you say so, then ok. You can disable this function.
I don't quite understand what you expect from me in this situation? As far as I understand your mvp, I will not do the slightest interference in the mvp code. Are you saying that you don't like any requests? Ok, remove them.
What do you need from me?)
14.06.2023, 15:33
Original comment available on version: ua
Please join the conversation. If you have something to say - please write a comment. You will need a mobile phone and an SMS code for identification to enter.
Log in and comment
Copyright © OneBox 2024
