All RPGs and Storygames by Tod Foley are now available at DrivethruRPG. Bring these games to your table!
This week's roundup includes a look at new Zachtronics title Eliza, the moral panic about Fortnite and 'good play', and the incredible Wii Sports speedrunners, among others. ...
Today we are going to learn how to migrate users into Drupal. The example code will be explained in two blog posts. In this one, we cover the migration of email, timezone, username, password, and status. In the next one, we will cover creation date, roles, and profile pictures. Several techniques will be implemented to ensure that the migrated data is valid. For example, making sure that usernames are not duplicated.
Although the example is standalone, we will build on many of the concepts that had already been covered in the series. For instance, a file migration is included to import images used as profile pictures. This topic has been explained in detail in a previous post, and the example code is pretty similar. Therefore, no explanation is provided about the file migration to keep the focus on the user migration. Feel free to read other posts in the series if you need a refresher.Getting the code
You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD users whose machine name is ud_migrations_users. The two migrations to execute are udm_user_pictures and udm_users. Notice that both migrations belong to the same module. Refer to this article to learn where the module should be placed.
The example assumes Drupal was installed using the standard installation profile. Particularly, we depend on a Picture (user_picture) image field attached to the user entity. The word in parenthesis represents the machine name of the image field.
The explanation below is only for the user migration. It depends on a file migration to get the profile pictures. One motivation to have two migrations is for the images to be deleted if the file migration is rolled back. Note that other techniques exist for migrating images without having to create a separate migration. We have covered two of them in the articles about subfields and constants and pseudofields.Understanding the source
It is very important to understand the format of your source data. This will guide the transformation process required to produce the expected destination format. For this example, it is assumed that the legacy system from which users are being imported did not have unique usernames. Emails were used to uniquely identify users, but that is not desired in the new Drupal site. Instead, a username will be created from a public_name source column. Special measures will be taken to prevent duplication as Drupal usernames must be unique. Two more things to consider. First, source passwords are provided in plain text (never do this!). Second, some elements might be missing in the source like roles and profile picture. The following snippet shows a sample record for the source section:source: plugin: embedded_data data_rows: - legacy_id: 101 public_name: 'Michele' user_email: 'email@example.com' timezone: 'America/New_York' user_password: 'totally insecure password 1' user_status: 'active' member_since: 'January 1, 2011' user_roles: 'forum moderator, forum admin' user_photo: 'P01' ids: legacy_id: type: integerConfiguring the destination and dependencies
The destination section specifies that user is the target entity. When that is the case, you can set an optional md5_passwords configuration. If it is set to true, the system will take an MD5 hashed password and convert it to the encryption algorithm that Drupal uses. For more information password migrations refer to these articles for basic and advanced use cases. To migrate the profile pictures, a separate migration is created. The dependency of user on file is added explicitly. Refer to these articles more information on migrating images and files and setting dependencies. The following code snippet shows how the destination and dependencies are set:destination: plugin: 'entity:user' md5_passwords: true migration_dependencies: required: - udm_user_pictures optional: Processing the fields
The interesting part of a user migration is the field mapping. The specific transformation will depend on your source, but some arguably complex cases will be addressed in the example. Let’s start with the basics: verbatim copies from source to destination. The following snippet shows three mappings:mail: user_email init: user_email timezone: user_timezone
The mail, init, and timezone entity properties are copied directly from the source. Both mail and init are email addresses. The difference is that mail stores the current email, while init stores the one used when the account was first created. The former might change if the user updates its profile, while the latter will never change. The timezone needs to be a string taken from a specific set of values. Refer to this page for a list of supported timezones.name: - plugin: machine_name source: public_name - plugin: make_unique_entity_field entity_type: user field: name postfix: _
The name, entity property stores the username. This has to be unique in the system. If the source data contained a unique value for each record, it could be used to set the username. None of the unique source columns (eg., legacy_id) is suitable to be used as username. Therefore, extra processing is needed. The machine_name plugin converts the public_name source column into transliterated string with some restrictions: any character that is not a number or letter will be converted to an underscore. The transformed value is sent to the make_unique_entity_field. This plugin makes sure its input value is not repeated in the whole system for a particular entity field. In this example, the username will be unique. The plugin is configured indicating which entity type and field (property) you want to check. If an equal value already exists, a new one is created appending what you define as postfix plus a number. In this example, there are two records with public_name set to Benjamin. Eventually, the usernames produced by running the process plugins chain will be: benjamin and benjamin_1.process: pass: plugin: callback callable: md5 source: user_password destination: plugin: 'entity:user' md5_passwords: true
The pass, entity property stores the user’s password. In this example, the source provides the passwords in plain text. Needless to say, that is a terrible idea. But let’s work with it for now. Drupal uses portable PHP password hashes implemented by PhpassHashedPassword. Understanding the details of how Drupal converts one algorithm to another will be left as an exercise for the curious reader. In this example, we are going to take advantage of a feature provided by the migrate API to automatically convert MD5 hashes to the algorithm used by Drupal. The callback plugin is configured to use the md5 PHP function to convert the plain text password into a hashed version. The last part of the puzzle is set, in the process section, the md5_passwords configuration to true. This will take care of converting the already md5-hashed password to the value expected by Drupal.
Note: MD5-hash passwords are insecure. In the example, the password is encrypted with MD5 as an intermediate step only. Drupal uses other algorithms to store passwords securely.status: plugin: static_map source: user_status map: inactive: 0 active: 1
The status, entity property stores whether a user is active or blocked from the system. The source user_status values are strings, but Drupal stores this data as a boolean. A value of zero (0) indicates that the user is blocked while a value of one (1) indicates that it is active. The static_map plugin is used to manually map the values from source to destination. This plugin expects a map configuration containing an array of key-value mappings. The value from the source is on the left. The value expected by Drupal is on the right.
Technical note: Booleans are true or false values. Even though Drupal treats the status property as a boolean, it is internally stored as a tiny int in the database. That is why the numbers zero or one are used in the example. For this particular case, using a number or a boolean value on the right side of the mapping produces the same result.
In the next blog post, we will continue with the user migration. Particularly, we will explain how to migrate the user creation time, roles, and profile pictures.
What did you learn in today’s blog post? Have you migrated user passwords before, either in plain text or hashed? Did you know how to prevent duplicates for values that need to be unique in the system? Were you aware of the plugin that allows you to manually map values from source to destination? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.
This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.
Module provides an api to simplifies building multi step forms. FormMultistepBase class provided by the module take care such things like managing steps, placing next, back and submit buttons at the specific steps (no matter how many steps you need to provide) etc. All you have to worry about with this class to build a multi step form is form.
In the module there is an example module that shows how to build an ajax multi step forms with class FormMultistepBase.
Today we continue the conversation about migration dependencies with a hierarchical taxonomy terms example. Along the way, we will present the process and syntax for migrating into multivalue fields. The example consists of two separate migrations. One to import taxonomy terms accounting for term hierarchy. And another to import into a multivalue taxonomy term field. Following this approach, any node and taxonomy term created by the migration process will be removed from the system upon rollback.Getting the code
You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD multivalue taxonomy terms whose machine name is ud_migrations_multivalue_terms. The two migrations to execute are udm_dependencies_multivalue_term and udm_dependencies_multivalue_node. Notice that both migrations belong to the same module. Refer to this article to learn where the module should be placed.
The example assumes Drupal was installed using the standard installation profile. Particularly, a Tags (tags) taxonomy vocabulary, an Article (article) content type, and a Tags (field_tags) field that accepts multiple values. The words in parenthesis represent the machine name of each element.Migrating taxonomy terms and their hierarchy
The example data for the taxonomy terms migration is fruits and fruit varieties. Each row will contain the name and description of the fruit. Additionally, it is possible to define a parent term to establish hierarchy. For example, “Red grape” is a child of “Grape”. Note that no numerical identifier is provided. Instead, the value of the name is used as a string identifier for the migration. If term names could change over time, it is recommended to have another column that did not change (e.g., an autoincrementing number). The following snippet shows how the source section is configured:source: plugin: embedded_data data_rows: - fruit_name: 'Grape' fruit_description: 'Eat fresh or prepare some jelly.' - fruit_name: 'Red grape' fruit_description: 'Sweet grape' fruit_parent: 'Grape' - fruit_name: 'Pear' fruit_description: 'Eat fresh or prepare a jam.' ids: fruit_name: type: string
The destination is quite short. The target entity is set to taxonomy terms. Additionally, you indicate which vocabulary to migrate into. If you have terms that would be stored in different vocabularies, you can use the vid property in the process section to assign the target vocabulary. If you write to a single one, the default_bundle key in the destination can be used instead. The following snippet shows how the destination section is configured:destination: plugin: 'entity:taxonomy_term' default_bundle: tags
For the process section, three entity properties are set: name, description, and parent. The first two are strings copied directly from the source. In the case of parent, it is an entity reference to another taxonomy term. It stores the taxonomy term id (tid) of the parent term. To assign its value, the migration_lookup plugin is configured similar to the previous example. The difference is that, in this case, the migration to reference is the same one being defined. This sets an important consideration. Parent terms should be migrated before their children. This way, they can be found by the lookup operation. Also note that the lookup value is the term name itself, because that is what this migration set as the unique identifier in the source section. The following snippet shows how the process section is configured:process: name: fruit_name description: fruit_description parent: plugin: migration_lookup migration: udm_dependencies_multivalue_term source: fruit_parent
Technical note: The taxonomy term entity contains other properties you can write to. For a list of available options check the baseFieldDefinitions() method of the Term class defining the entity. Note that more properties can be available up in the class hierarchy.Migrating multivalue taxonomy terms fields
The next step is to create a node migration that can write to a multivalue taxonomy term field. To stay on point, only one more field will be set: the title, which is required by the node entity. Read this change record for more information on how the Migrate API processes Entity API validation. The following snippet shows how the source section is configured for the node migration:source: plugin: embedded_data data_rows: - unique_id: 1 thoughtful_title: 'Amazing recipe' fruit_list: 'Green apple, Banana, Pear' - unique_id: 2 thoughtful_title: 'Fruit-less recipe' ids: unique_id: type: integer
The fruits column contains a comma separated list of taxonomies to apply. Note that the values match the unique identifiers of the taxonomy term migration. If you had used numbers as migration identifiers there, you would have to use those numbers in this migration to refer to the terms. An example of that was presented in the previous post. Also note that there is one record that has no terms associated. This will be considered during the field mapping. The following snippet shows how the process section is configured for the node migration:process: title: thoughtful_title field_tags: - plugin: skip_on_empty source: fruit_list method: process message: 'No fruit_list listed.' - plugin: explode delimiter: ',' - plugin: migration_lookup migration: udm_dependencies_multivalue_term
The title of the node is a verbatim copy of the thoughtful_title column. The Tags fields, mapped using its machine name field_tags, uses three chained process plugins. The skip_on_empty plugin reads the value of the fruit_list column and skips the processing of this field if no value is provided. This is done to accommodate the fact that some records in the source do not specify tags. Note that the method configuration key is set to process. This indicates that only this field should be skipped and not the entire record. Ultimately, tags are optional in this context and nodes should still be imported even if no tag is associated.
The explode plugin allows you to break a string into an array, using a delimiter to determine where to make the cut. Later, this array is passed to the migration_lookup plugin specifying the term migration as the one to use for the lookup operation. Again, the taxonomy term names are used here because they are the unique identifiers of the term migration. Note that neither of these plugins has a source configuration. This is because when process plugins are chained, the result of one plugin is sent as source to be transformed by the next one in line. The end result is an array of taxonomy term ids that will be assigned to field_tags. The migration_lookup is able to process single values and arrays.
The last part of the migration is specifying the process section and any dependencies. Refer to this article for more details on setting migration dependencies. The following snippet shows how both are configured for the node migration:destination: plugin: 'entity:node' default_bundle: article migration_dependencies: required: - udm_dependencies_multivalue_term optional: More syntactic sugar
One way to set multivalue fields in Drupal migrations is assigning its value to an array. Another option is to set each value manually using field deltas. Deltas are integer numbers starting with zero (0) and incrementing by one (1) for each element of a multivalue field. Although you could set any delta in the Migrate API, consider the field definition in Drupal. It is possible that limits had been set to the number of values a field could hold. You can specify deltas and subfields at the same time. The full syntax is field_name/field_delta/subfield. The following example shows the syntax for a multivalue image field:process: field_photos/0/target_id: source_fid_first field_photos/0/alt: source_alt_first field_photos/1/target_id: source_fid_second field_photos/1/alt: source_alt_second field_photos/2/target_id: source_fid_third field_photos/2/alt: source_alt_third
Manually setting a multivalue field is less flexible and error-prone. In today’s example, we showed how to accommodate for the list of terms not being provided. Imagine having to that for each delta and subfield combination, but the functionality is there in case you need it. In the end, Drupal offers more syntactic sugar so you can write shorted field mappings. Additionally, there are various process plugins that can handle arrays for setting multivalue fields.
Note: There are other ways to migrate multivalue fields. For example, when using the entity_generate plugin provided by Migrate Plus, there is no need to create a separate taxonomy term migration. This plugin is able to create the terms on the fly while running the import process. The caveat is that terms created this way are not deleted upon rollback.
What did you learn in today’s blog post? Have you ever done a taxonomy term migration before? Were you aware of how to migrate hierarchical entities? Did you know you can manually import multivalue fields using deltas? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.
This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether the migration series or other topics.
While it needs some editing to clean up typos in the table I really do like the concept. Looking forward ro playing with it. Well worth a buck or two.
ESPN and ABC have delayed the broadcast of an Apex Legends tournament that was set to air this weekend, a decision ABC says was made out of respect to those impacted by recent mass shootings in the US. ...
Last week I recapped my experience at Decoupled Days 2019, and there's one exciting development coming out of that event I wanted to write more about. We've had a serious challenge using the JSON:API module with Drupal Commerce for a while now – cross-bundle entity collections.
Drupal Commerce lets merchants define any number of product types to match their product data models. However, these different types of products all need to be represented in a single product catalog. Building that is easy enough with Drupal's Views module, but it's a challenge via JSON:API given it's oriented more toward serving entities of a single type, or bundle, from its collection resources.
Kubernetes is taking over the world of infrastructure management, at least for larger-scale operations, and best practices have started to solidify. One of those best practices is the cultivation of Custom Resource Definitions (CRDs) to describe your applications in a Kubernetes-native way, and Operators to manage your the Custom Resources running on your Kubernetes clusters.
In the Drupal community, Kubernetes uptake has been somewhat slow, but is on the rise. Just like with Docker adoption for local development, the tooling and documentation has been slowly percolating. For example, Tess Flynn from TEN7 has been boldly going where no one has gone before (oops, wrong scifi series!) using the Force to promote Drupal usage in a Kubernetes environment.
Alter page cache middleware class to ignore query params.Motivation
Time to time we are faced with necessary to ignore some query params to use page cache. See https://www.drupal.org/project/drupal/issues/2062463 for example.
Via this module you can ignore params like fbclid or gclid
Just add params to config page_cache_query_ignore.settings like:
This module provide payment method via OnePAY.VN for the Drupal Commerce payment system.
This module is developed base on OnePAY.VN document. If you have any problem please review the document at developer page.Dependencies